Module benchmark 

Benchmark metadata derived from a benchmark’s Dockerfile.

Functions

is_per_task

A benchmark is per-task — one eval image baked per task (swe-bench-style) — iff its Dockerfile declares LABEL eval.benchmark.env="per-task". That label is the single source of truth for per-task detection across the CLI (build, run, oracle) and the chart’s perTask value (benchmarks/RULES.md 24f). Matched on a LABEL line so a comment or RUN echo mentioning the string cannot false-positive.

is_per_task_by_name

is_per_task for a benchmark by name — reads benchmarks/<name>/Dockerfile (a missing file reads as shared-env, false).