Skip to main content

Module naming

Module naming 

Source
Expand description

Image-name and bake-target conventions — the single source of truth for how the CLI maps a (benchmark, agent, model) axis onto registry image refs and docker buildx bake target names.

These were previously inlined as format! strings scattered across build.rs, run.rs, inspect.rs, and images.rs. Centralizing them here makes the conventions testable (see the unit tests below) and keeps every call site in agreement — the contract that benchmarks/agents/models are built and run against (src/RULES.md, benchmarks/RULES.md rule 24).

Functions§

agent_bake_target
Bake target for an agent: agent-<name>.
agent_image
{registry}/agents/<name>:<tag> — the per-agent image.
benchmark_bake_target
Bake target for a benchmark: benchmark-<name>.
benchmark_image
{registry}/benchmarks/<name>:<tag> — the per-benchmark base image.
benchmark_task_image
{registry}/benchmarks/<name>-<task>:<tag> — a per-task benchmark variant (swe-bench-style), built outside bake’s static graph (BAKE.md). The task id is lowercased: Docker repo names + tags MUST be lowercase and some per-task ids carry uppercase (swe-bench-pro’s instance_NodeBB__…). This affects only the image NAME — EVAL_TASK_ID keeps the real id at runtime.
compose_artifact
{registry}/compose/<name>:latest — a benchmark’s published compose file.
eval_image
{registry}/evals/<benchmark>--<agent>:<tag> — the combined eval image (shared-env benchmarks: one image, task chosen at runtime). The -- separator is load-bearing: the OpenShift flattener collapses it to a single - for imagestream names (see flatten_imagestream).
eval_task_image
{registry}/evals/<benchmark>-<task>--<agent>:<tag> — the combined eval image for a per-task benchmark (each task bakes a separate image; the task id is part of the name, mirroring benchmark_task_image). Every surface (build / compose / container / job) MUST address per-task evals by this name (benchmarks/RULES.md — eval-image naming). The task id is lowercased for the same Docker-tag reason as benchmark_task_image.
flatten_imagestream
Nested image repo path → OpenShift imagestream name (single segment). core/gateways keep their prefix (core/otel → core-otel); the per-eval categories drop it (benchmarks/aime → aime, evals/aime--codex → aime-codex); dots and -- collapse to -.
model_bake_target
Bake target for a model: model-<name>, with . → _ because HCL target names can’t contain dots (e.g. gpt-5.4 → model-gpt-5_4).
model_image
{registry}/models/<model>:<tag> — the per-model gateway image.
release_name
Sanitize an axis-derived string into a valid Helm release name — a DNS-1123 label: lowercase, each run of non-[a-z0-9] collapses to one -, no leading or trailing -, capped at Helm’s 53-char limit. Job mode’s release name is <benchmark>-<agent>-task-<task>; per-task task ids carry chars Helm forbids (SWE-bench’s sympy__sympy-24066 has _), so without this run --mode job can’t even render the chart for a per-task benchmark. Sibling to flatten_imagestream — both make a name k8s-safe.