Skip to main content

Crate visual_rubric

Crate visual_rubric 

Source
Expand description

Shared AI visual-rubric runner for screenshot review.

This crate owns the Codex ACP plumbing so browser screenshots, offscreen renderer captures, and VM/VNC screenshots can use one rubric path.

Direct Codex ACP evaluation sends the rubric prompt and screenshot in one multimodal request. The crate also provides a two-stage pipeline: vision model extraction via an OpenAI-compatible HTTP API, then rubric scoring via ACP.

Re-exports§

pub use calibration::CALIBRATION_CORPUS_SCHEMA_VERSION;
pub use calibration::CalibrationCaptureV1;
pub use calibration::CalibrationCorpusV1;
pub use calibration::REQUIRED_CALIBRATION_CATEGORIES;
pub use cli::Cli;
pub use coverage::COVERAGE_CONTRACT_SCHEMA_VERSION;
pub use coverage::COVERAGE_REPORT_SCHEMA_VERSION;
pub use coverage::CoverageContractV1;
pub use coverage::CoverageExclusionV1;
pub use coverage::CoverageReportV1;
pub use coverage::CoverageSurfaceV1;
pub use coverage::CoverageTransitionV1;
pub use deterministic::DETERMINISTIC_EVIDENCE_SCHEMA_VERSION;
pub use deterministic::DeterministicEvidenceV1;
pub use deterministic::DeterministicObservationV1;
pub use manifest::ArtifactDigest;
pub use manifest::CAPTURE_MANIFEST_SCHEMA_VERSION;
pub use manifest::CaptureCell;
pub use manifest::CaptureEnvironment;
pub use manifest::CaptureManifest;
pub use manifest::ManifestError;
pub use manifest::Viewport;
pub use qa::CALIBRATION_SENTINEL_SCHEMA_VERSION;
pub use qa::CalibrationExpectation;
pub use qa::CalibrationSentinelV1;
pub use qa::FINDING_SCHEMA_VERSION;
pub use qa::FindingV1;
pub use qa::OBSERVATION_SCHEMA_VERSION;
pub use qa::ObservationV1;
pub use qa::VISUAL_RUN_REPORT_SCHEMA_VERSION;
pub use qa::VisualRunReportV1;
pub use qa::VisualRunStatus;
pub use qa::finding_fingerprint;
pub use qa::validate_calibration_sentinels;
pub use qa::validate_visual_run;
pub use qa::validate_visual_run_against_manifest;

Modules§

calibration
Checked-in calibration corpus contracts for visual-detector QA.
cli
Command-line interface and audit report types.
coverage
Shared semantic coverage contracts for autonomous visual QA.
deterministic
Deterministic browser evidence used before semantic rubric evaluation.
manifest
Versioned capture manifests emitted by project-owned visual producers.
presets
Question preset interface for standardized rubric questions.
qa
Portable, versioned contracts for autonomous visual QA evidence.
vision
OpenAI-compatible API client for the vision and rubric stages.

Structs§

AssetRubricReport
Per-asset report item.
AssetSnapshot
Content-hash snapshot for a caller-provided asset set.
BatchRubricConfig
Batch rubric runner configuration.
BatchRubricReport
Serializable batch rubric report.
BatchRubricRun
Generic batch runner around RubricPool.
IssueClassificationInput
Input passed to caller-provided issue classifiers.
IssueRecommendation
Caller-provided issue recommendation.
LogCaptureConfig
Directories used to capture logs produced by Codex ACP.
PageResult
Result from evaluating one page screenshot.
PoolConfig
Configuration for a crate::RubricPool.
PoolStats
Snapshot of pool execution counters.
RateLimitEvent
One observed ACP rate-limit event.
RubricEffort
Validated Codex ACP reasoning effort value.
RubricOptions
Optional model settings for one rubric request.
RubricPool
Reusable worker pool for evaluating screenshot rubrics through Codex ACP.
RubricReport
LLM-optimized report aggregating multiple page evaluations.
RubricRunConfig
Runtime configuration for rubric evaluation.
RubricVerdict
Parsed rubric verdict returned by ACP.
RubricVerdictStatus
Validated rubric verdict status.
SequenceFrame
One ordered screenshot checkpoint in an interaction journey.
SequenceOptions
Policy applied to one ordered screenshot sequence.
TomlConfig
Full shape of ~/.config/visual-rubric/config.toml.
TomlRubric
[rubric] section of the TOML config.
TomlSequence
[sequence] ordered-checkpoint policy.
TomlVision
[vision] section of the TOML config.

Enums§

AggregateStatus
Overall status for a batch rubric report.
AssetChange
Content change for one asset between two snapshots.
AssetRubricResult
Per-asset rubric result.
ConfigMode
Backend mode read from config.toml.
LogPathMode
Path representation for copied ACP logs in batch reports.
PoolError
Errors produced by the ACP pool, worker runtime, or vision API.
RecommendationSeverity
Severity for a caller-provided recommendation.
RubricError
Errors returned by public one-shot rubric APIs.
SelectionMode
Asset selection mode for a batch rubric run.

Constants§

DEFAULT_CODEX_ACP_MODEL
Default Codex ACP model.
DEFAULT_CODEX_ACP_REASONING_EFFORT
Default Codex ACP reasoning effort.
DEFAULT_SYSTEM_PROMPT
Default system prompt used for screenshot rubric requests.
DEFAULT_VISION_PROMPT
Default prompt for the vision extraction stage.

Traits§

IssueClassifier
Optional caller hook for mapping failed/error assets to reusable recommendations.

Functions§

assert_image_rubric
Evaluates a PNG and returns an error when the verdict is not pass.
assert_verdict
Converts a verdict into an assertion-style result.
default_codex_acp_binary
Returns the default Codex ACP executable name.
default_options
Returns the default rubric options.
diff_snapshots
Compares two asset snapshots.
direct_codex_gpt_config
Build the canonical direct Codex GPT TOML configuration.
encode_png
Reads and base64-encodes a PNG file.
evaluate_configured
Load configuration from ~/.config/visual-rubric/config.toml and evaluate png_path against question.
evaluate_configured_with_vision
Like evaluate_configured but also returns the vision model’s text description alongside the rubric verdict.
evaluate_image_rubric
Evaluates a PNG with default options.
evaluate_image_rubric_pipeline
Two-stage pipeline evaluation: vision model → rubric model.
evaluate_image_rubric_pipeline_with_vision
Like evaluate_image_rubric_pipeline but also returns the vision model’s text description alongside the rubric verdict.
evaluate_image_rubric_with_config
Evaluates a PNG with caller-provided model and runtime configuration.
evaluate_image_rubric_with_options
Evaluates a PNG with caller-provided model options.
evaluate_image_sequence_rubric_pipeline
Two-stage evaluation for an ordered interaction sequence.
evaluate_image_sequence_rubric_pipeline_with_options
Two-stage sequence evaluation with an explicit checkpoint policy.
evaluate_image_sequence_rubric_with_config
Evaluates an ordered screenshot sequence as one interaction journey.
evaluate_image_sequence_rubric_with_options
Evaluates an ordered screenshot sequence with an explicit checkpoint policy and ACP runtime configuration.
load_config_toml
Load the TOML config from path, falling back to the user config path and then the Infernix-managed /etc/visual-rubric/config.toml.
parse_verdict
Parses strict rubric JSON into a typed verdict.
run
Runs the CLI command.
select_changed
Selects the assets that should be evaluated from a snapshot diff.