Expand description
Shared AI visual-rubric runner for screenshot review.
This crate owns the Codex ACP plumbing so browser screenshots, offscreen renderer captures, and VM/VNC screenshots can use one rubric path.
Direct Codex ACP evaluation sends the rubric prompt and screenshot in one multimodal request. The crate also provides a two-stage pipeline: vision model extraction via an OpenAI-compatible HTTP API, then rubric scoring via ACP.
Re-exports§
pub use calibration::CALIBRATION_CORPUS_SCHEMA_VERSION;pub use calibration::CalibrationCaptureV1;pub use calibration::CalibrationCorpusV1;pub use calibration::REQUIRED_CALIBRATION_CATEGORIES;pub use cli::Cli;pub use coverage::COVERAGE_CONTRACT_SCHEMA_VERSION;pub use coverage::COVERAGE_REPORT_SCHEMA_VERSION;pub use coverage::CoverageContractV1;pub use coverage::CoverageExclusionV1;pub use coverage::CoverageReportV1;pub use coverage::CoverageSurfaceV1;pub use coverage::CoverageTransitionV1;pub use deterministic::DETERMINISTIC_EVIDENCE_SCHEMA_VERSION;pub use deterministic::DeterministicEvidenceV1;pub use deterministic::DeterministicObservationV1;pub use manifest::ArtifactDigest;pub use manifest::CAPTURE_MANIFEST_SCHEMA_VERSION;pub use manifest::CaptureCell;pub use manifest::CaptureEnvironment;pub use manifest::CaptureManifest;pub use manifest::ManifestError;pub use manifest::Viewport;pub use qa::CALIBRATION_SENTINEL_SCHEMA_VERSION;pub use qa::CalibrationExpectation;pub use qa::CalibrationSentinelV1;pub use qa::FINDING_SCHEMA_VERSION;pub use qa::FindingV1;pub use qa::OBSERVATION_SCHEMA_VERSION;pub use qa::ObservationV1;pub use qa::VISUAL_RUN_REPORT_SCHEMA_VERSION;pub use qa::VisualRunReportV1;pub use qa::VisualRunStatus;pub use qa::finding_fingerprint;pub use qa::validate_calibration_sentinels;pub use qa::validate_visual_run;pub use qa::validate_visual_run_against_manifest;
Modules§
- calibration
- Checked-in calibration corpus contracts for visual-detector QA.
- cli
- Command-line interface and audit report types.
- coverage
- Shared semantic coverage contracts for autonomous visual QA.
- deterministic
- Deterministic browser evidence used before semantic rubric evaluation.
- manifest
- Versioned capture manifests emitted by project-owned visual producers.
- presets
- Question preset interface for standardized rubric questions.
- qa
- Portable, versioned contracts for autonomous visual QA evidence.
- vision
- OpenAI-compatible API client for the vision and rubric stages.
Structs§
- Asset
Rubric Report - Per-asset report item.
- Asset
Snapshot - Content-hash snapshot for a caller-provided asset set.
- Batch
Rubric Config - Batch rubric runner configuration.
- Batch
Rubric Report - Serializable batch rubric report.
- Batch
Rubric Run - Generic batch runner around
RubricPool. - Issue
Classification Input - Input passed to caller-provided issue classifiers.
- Issue
Recommendation - Caller-provided issue recommendation.
- LogCapture
Config - Directories used to capture logs produced by Codex ACP.
- Page
Result - Result from evaluating one page screenshot.
- Pool
Config - Configuration for a
crate::RubricPool. - Pool
Stats - Snapshot of pool execution counters.
- Rate
Limit Event - One observed ACP rate-limit event.
- Rubric
Effort - Validated Codex ACP reasoning effort value.
- Rubric
Options - Optional model settings for one rubric request.
- Rubric
Pool - Reusable worker pool for evaluating screenshot rubrics through Codex ACP.
- Rubric
Report - LLM-optimized report aggregating multiple page evaluations.
- Rubric
RunConfig - Runtime configuration for rubric evaluation.
- Rubric
Verdict - Parsed rubric verdict returned by ACP.
- Rubric
Verdict Status - Validated rubric verdict status.
- Sequence
Frame - One ordered screenshot checkpoint in an interaction journey.
- Sequence
Options - Policy applied to one ordered screenshot sequence.
- Toml
Config - Full shape of
~/.config/visual-rubric/config.toml. - Toml
Rubric [rubric]section of the TOML config.- Toml
Sequence [sequence]ordered-checkpoint policy.- Toml
Vision [vision]section of the TOML config.
Enums§
- Aggregate
Status - Overall status for a batch rubric report.
- Asset
Change - Content change for one asset between two snapshots.
- Asset
Rubric Result - Per-asset rubric result.
- Config
Mode - Backend mode read from
config.toml. - LogPath
Mode - Path representation for copied ACP logs in batch reports.
- Pool
Error - Errors produced by the ACP pool, worker runtime, or vision API.
- Recommendation
Severity - Severity for a caller-provided recommendation.
- Rubric
Error - Errors returned by public one-shot rubric APIs.
- Selection
Mode - Asset selection mode for a batch rubric run.
Constants§
- DEFAULT_
CODEX_ ACP_ MODEL - Default Codex ACP model.
- DEFAULT_
CODEX_ ACP_ REASONING_ EFFORT - Default Codex ACP reasoning effort.
- DEFAULT_
SYSTEM_ PROMPT - Default system prompt used for screenshot rubric requests.
- DEFAULT_
VISION_ PROMPT - Default prompt for the vision extraction stage.
Traits§
- Issue
Classifier - Optional caller hook for mapping failed/error assets to reusable recommendations.
Functions§
- assert_
image_ rubric - Evaluates a PNG and returns an error when the verdict is not pass.
- assert_
verdict - Converts a verdict into an assertion-style result.
- default_
codex_ acp_ binary - Returns the default Codex ACP executable name.
- default_
options - Returns the default rubric options.
- diff_
snapshots - Compares two asset snapshots.
- direct_
codex_ gpt_ config - Build the canonical direct Codex GPT TOML configuration.
- encode_
png - Reads and base64-encodes a PNG file.
- evaluate_
configured - Load configuration from
~/.config/visual-rubric/config.tomland evaluatepng_pathagainstquestion. - evaluate_
configured_ with_ vision - Like
evaluate_configuredbut also returns the vision model’s text description alongside the rubric verdict. - evaluate_
image_ rubric - Evaluates a PNG with default options.
- evaluate_
image_ rubric_ pipeline - Two-stage pipeline evaluation: vision model → rubric model.
- evaluate_
image_ rubric_ pipeline_ with_ vision - Like
evaluate_image_rubric_pipelinebut also returns the vision model’s text description alongside the rubric verdict. - evaluate_
image_ rubric_ with_ config - Evaluates a PNG with caller-provided model and runtime configuration.
- evaluate_
image_ rubric_ with_ options - Evaluates a PNG with caller-provided model options.
- evaluate_
image_ sequence_ rubric_ pipeline - Two-stage evaluation for an ordered interaction sequence.
- evaluate_
image_ sequence_ rubric_ pipeline_ with_ options - Two-stage sequence evaluation with an explicit checkpoint policy.
- evaluate_
image_ sequence_ rubric_ with_ config - Evaluates an ordered screenshot sequence as one interaction journey.
- evaluate_
image_ sequence_ rubric_ with_ options - Evaluates an ordered screenshot sequence with an explicit checkpoint policy and ACP runtime configuration.
- load_
config_ toml - Load the TOML config from
path, falling back to the user config path and then the Infernix-managed/etc/visual-rubric/config.toml. - parse_
verdict - Parses strict rubric JSON into a typed verdict.
- run
- Runs the CLI command.
- select_
changed - Selects the assets that should be evaluated from a snapshot diff.