Expand description
harn eval prompt <file> --fleet <models> — render and optionally run
a single .harn.prompt template across a fleet of models so authors
can validate the capability-adapted envelope per model side-by-side.
Render mode is the v1 acceptance path: it pushes an LLM render
context per model, calls the template engine, and emits the rendered
envelope plus a wire-format diff. Run and judge modes synthesize a
tiny Harn driver and route through the existing execute_run
pipeline so credentialed LLM calls, mock fixtures, and the
LlmRenderContext injection all stay on the canonical path.
§.harn dispatch
The aggregation layer (fleet resolution, per-model rendering via
LlmRenderContext, run/judge fanout, context-fixture evaluation)
stays in Rust — it reaches into harn_vm::stdlib::template,
harn_vm::llm_config, and harn_vm::orchestration internals that
aren’t exposed to script-land today.
The rendering layer (terminal / JSON / HTML) is delegated to
crates/harn-stdlib/src/stdlib/cli/eval/prompt.harn. The Rust shim
serialises the assembled PromptReport to JSON and forwards it via
[PROMPT_REPORT_ENV] plus a couple of mode env vars, then routes
through the standard dispatch wedge. The script just reads the
report, picks a formatter, and emits the payload (or writes it to
--out-file).