Crate loadmaster_probe 

Tool-call and structured-output conformance probe, eval runner.

run::probe runs declarative fixtures against any OpenAI-compatible endpoint and classifies each failure; eval runs a user’s prompts and stores results in SQLite.

Re-exports

pub use client::Client;

pub use fixture::Fixture;

pub use run::Options;

pub use run::Outcome;

pub use run::PROBE_VERSION;

pub use run::Report;

Modules

check

Condition evaluation over captured exchanges. Pure: no I/O, clock or randomness.

client

OpenAI-compatible client with raw capture and streaming reassembly.

compare

probe compare: two eval runs side by side, with significance for the proportions.

db

Eval results in SQLite (~/.loadmaster/eval.db).

detect

Which runtime is behind an OpenAI-compatible URL, from its identifying endpoints.

embed

The embedding check (probe --embeddings, and up --verify for embedding models): /v1/embeddings returns finite vectors of one length, the same text gives the same vector every time, and a batch of two texts gives two different vectors.

eval

probe eval: the user’s prompts against one or more models, deterministic sampling.

fake

A scriptable fake OpenAI-compatible server for tests (and loadmaster __fake-runtime).

fixture

Declarative probe fixtures: YAML with request steps, a pass condition and ordered failure classes. Adding a fixture is adding a file; see docs/fixtures.md.

known

Known upstream issues: when a failure matches a curated signature, the probe prints the issue URL. The curated list is known_issues.toml next to the fixtures.

run

Execute fixtures against an endpoint and reduce them to a verdict.

time

UTC timestamps for the write boundary (SQLite rows, lockfile verdicts). Nothing else in the probe reads the clock.