Skip to main content

Module verify

Module verify 

Source
Expand description

Decision-comment verification for omni-dev ai jev verify-decision (#1779).

Splits a maintainer’s decision comment into single, checkable statements (via the configured AI backend), then checks each one against the source it cites with one noul question per statement, one Jev call per cited source. This is the wording and grouping validated in #1779’s per-statement experiment (E4): checking one statement at a time against its source accepted 5/5 accurate claims and 0/20 wrong ones, where checking a whole comment at once let overstatements through.

A coverage check (new, unvalidated) additionally asks whether the split statements together say everything the comment claims, guarding against a splitter that drops or strengthens a claim.

Structs§

CommentRef
Identifies the comment that was checked.
ReportModels
The models that answered.
ReportUsage
Token usage summed over every call.
Source
One cited source: a cited issue plus the pull requests that closed it, or a standalone cited pull request.
SourceReport
One source that was (or was meant to be) checked against.
VerifiedStatement
One statement’s verification outcome.
VerifyOptions
Knobs for run_verify.
VerifyReport
The result of verifying one decision comment.

Enums§

CommentSelector
Which decision comment verify-decision should check.
Verdict
The result of checking one decision comment.

Constants§

DEFAULT_COVERAGE
Default threshold below which low coverage is a review reason.
DEFAULT_REJECT_BELOW
Default threshold below which a statement rejects the whole comment.
DEFAULT_SUPPORTED
Default threshold above which a statement counts as supported.
MAX_STATEMENTS
Most statements one comment may be split into, before the splitter’s output is treated as broken rather than trusted.

Functions§

fetch_verify_input
Fetches everything verify-decision needs for one issue.
group_sources
Groups a decision comment’s citations into Sources.
parse_comment_selector
Parses --comment’s value: latest, a bare numeric id, or a full issue-comment URL (...#issuecomment-<id>, as GitHub links a comment).
run_verify
Verifies one decision comment against the sources it cites.
select_comment
Selects the comment verify-decision should check.