Module model 

Pinned, reproducible model adapter (#234).

The harness talks to a model through one small ModelRunner trait. Two real implementations ship:

• OpenAiRunner — a synchronous OpenAI-compatible chat client (OpenAI, Azure OpenAI, vLLM, llama.cpp, Ollama’s /v1 …). Decoding is pinned (temperature = 0, fixed seed) so a compliant provider is as deterministic as it can be.
• RecordedRunner — strict replay of responses previously captured from a real provider. Missing keys are a hard error (never a silent fallback), which is what makes CI runs and the determinism digest byte-stable across machines.

RecordingRunner wraps any real runner and captures every response so an operator can produce a replay file once with eval ab --record. Secrets (API keys) never enter a fingerprint, recording, or report.

Structs

ModelFingerprint

Identifies exactly which model + params produced a set of answers. Embedded in the report and the determinism digest so a third party knows precisely what was run.

ModelParams

Decoding parameters pinned for reproducibility.

ModelRequest

A single chat request (one system + one user turn). Deliberately minimal so the recording key is stable across runs.

ModelResponse

The model’s answer plus a content digest for auditing.

OpenAiRunner

Synchronous OpenAI-compatible chat client. Credentials are held only in memory.

RecordedRunner

Strict replay runner: every request must hit a recorded entry, or it errors. This guarantees the run is fully deterministic and machine-independent.

Recording

A captured set of real responses, keyed by ModelRequest::key. Serialized as JSON with a BTreeMap so the on-disk form is byte-stable.

RecordingRunner

Wraps a real runner and captures every response so it can be replayed later. Pass-through: the wrapped runner’s answers are returned unchanged.

Constants

PROVIDER_OPENAI

Provider label for the OpenAI-compatible chat runner.

PROVIDER_RECORDED

Provider label for the strict replay runner.

Traits

ModelRunner

Anything that can turn a request into a response under a fixed fingerprint.