Skip to main content

Module benchctx

Module benchctx 

Source
Expand description

Context benchmark (docs/TEST_PLAN.md §84–87, M0/M4): score task packs against the ground-truth corpus in benchmarks/tasks.json.

Metrics:

  • recall: ground-truth entities surfaced in the pack / total ground truth
  • precision: ground-truth entities / pack entities of ground-truth kinds
  • localization: ground-truth files mentioned in the pack content
  • hallucination: nonexistent entities that must NOT surface
  • budget: pack under the configured token budget

Structs§

BenchSummary
BenchTask
BenchmarkCorpus
GroundTruth
Hallucination
TaskResult

Functions§

locate_fixtures_dir
Locate the fixtures directory: walk up from cwd; fall back to the build-time manifest path (dev tooling).
print_summary
run_context_benchmark
Run the full context benchmark against benchmarks/tasks.json.
score_task_public