Crate llm_verify 

Black-box verification for LLM API endpoints, as a library.

The llm-verify binary is a thin CLI over this crate. Everything it does is reachable here, so a caller embedding the engine gets the same probes, the same order and the same verdict as the published tool — which is what lets it claim the two agree.

use llm_verify::{engine, probes::Cancel, Endpoint};

let cfg = engine::RunConfig::new(Endpoint {
    base_url: "https://api.anthropic.com".into(),
    api_key: std::env::var("ANTHROPIC_API_KEY").unwrap_or_default(),
    model: "claude-opus-4-5".into(),
    ..Default::default()
});
let report = engine::run(cfg, &Cancel::new(), &mut |_| {}).await?;
println!("{:?} {}", report.verdict.authenticity, report.verdict.score);

Probing something you reach through a relay

Half the suite asks questions about the endpoint — its error envelopes, its response headers, the token counts it reports. Those answers describe whichever hop is nearest the caller, so behind a relay they describe the relay. Ask for probes::Selection::model_only and only the steps whose evidence is the generated text itself will run. See probes::Subject.

let cfg = engine::RunConfig::new(endpoint)
    .model_only()
    .depth(llm_verify::probes::Depth::Fast)
    // Chosen by the caller, recorded in the report, so a contested verdict
    // can be replayed probe for probe.
    .seed(0x5EED);
let report = engine::run(cfg, &Cancel::new(), &mut |_| {}).await?;

Re-exports

pub use client::Endpoint;

pub use client::RequestOpts;

pub use engine::run;

pub use engine::RunConfig;

pub use i18n::Lang;

pub use probes::Cancel;

pub use probes::Depth;

pub use probes::Event;

pub use probes::Pace;

pub use probes::Selection;

pub use probes::Subject;

pub use protocol::Protocol;

pub use report::Authenticity;

pub use report::Channel;

pub use report::ProbeResult;

pub use report::Report;

pub use report::Status;

pub use report::Verdict;

Modules

client

HTTP transport. Deliberately low-level: several probes work by removing things a well-behaved client would always send (the auth header, the API version header) or by sending a body that is not valid JSON, so nothing here may quietly normalise a request on our behalf.

engine

One entry point for a whole verification run.

html

Self-contained HTML report.

i18n

Bilingual output.

pricing

Built-in list price table, USD per million tokens.

probes

Probe registry and the shared context every probe writes into.

protocol

Wire-format abstraction over the two chat protocols that matter in the resale market: OpenAI /v1/chat/completions and Anthropic /v1/messages.

report

The data model every probe writes into and every output format reads from.

util

Small self-contained helpers. Deliberately dependency-free: every one of these would otherwise pull in a crate (chrono, rand, tiktoken) that costs more binary size than the handful of lines it replaces.

verdict

Turns probe results into a conclusion.

Macros

t

Pick between an English and a Chinese message, applying format arguments to whichever is selected.

ts

Like t! but yields a &'static str, for cases that must not allocate or that feed APIs expecting a borrowed string.