Expand description
Decision-comment verification for omni-dev ai jev verify-decision (#1779).
Splits a maintainer’s decision comment into single, checkable statements
(via the configured AI backend), then checks each one against the
source it cites with one noul question per statement, one Jev call per
cited source. This is the wording and grouping validated in #1779’s
per-statement experiment (E4): checking one statement at a time against
its source accepted 5/5 accurate claims and 0/20 wrong ones, where
checking a whole comment at once let overstatements through.
A coverage check (new, unvalidated) additionally asks whether the split statements together say everything the comment claims, guarding against a splitter that drops or strengthens a claim.
Structs§
- Comment
Ref - Identifies the comment that was checked.
- Report
Models - The models that answered.
- Report
Usage - Token usage summed over every call.
- Source
- One cited source: a cited issue plus the pull requests that closed it, or a standalone cited pull request.
- Source
Report - One source that was (or was meant to be) checked against.
- Verified
Statement - One statement’s verification outcome.
- Verify
Options - Knobs for
run_verify. - Verify
Report - The result of verifying one decision comment.
Enums§
- Comment
Selector - Which decision comment
verify-decisionshould check. - Verdict
- The result of checking one decision comment.
Constants§
- DEFAULT_
COVERAGE - Default threshold below which low coverage is a review reason.
- DEFAULT_
REJECT_ BELOW - Default threshold below which a statement rejects the whole comment.
- DEFAULT_
SUPPORTED - Default threshold above which a statement counts as supported.
- MAX_
STATEMENTS - Most statements one comment may be split into, before the splitter’s output is treated as broken rather than trusted.
Functions§
- fetch_
verify_ input - Fetches everything
verify-decisionneeds for one issue. - group_
sources - Groups a decision comment’s citations into
Sources. - parse_
comment_ selector - Parses
--comment’s value:latest, a bare numeric id, or a full issue-comment URL (...#issuecomment-<id>, as GitHub links a comment). - run_
verify - Verifies one decision comment against the sources it cites.
- select_
comment - Selects the comment
verify-decisionshould check.