Struct LlmRuntime 

pub struct LlmRuntime {
    pub enabled: bool,
    pub api_style: String,
    pub unpriced: UnpricedPolicy,
    /* private fields */
}

The compiled LLM runtime: whether metering is on, the API style, and the price book. Built once per config (re)load and carried on the proxy Runtime.

Fields

enabled: bool

api_style: String

Request/response wire format. Only "openai" is understood today; anything else still meters (the OpenAI shape is a superset of most), but is recorded for forward-compat.

unpriced: UnpricedPolicy

What to do with a request for a model absent from the price book (count / block).

Implementations

impl LlmRuntime

pub fn build(cfg: &LlmCfg) -> Self

Compile an LlmRuntime from config. USD-per-million floats become integer micro-dollars per million; a negative price is clamped to 0 (free) rather than rejected, so a typo never stops the proxy booting. An invalid on_unpriced_model falls back to count (never a hard boot failure) — the value is re-validated at config load where a typo is surfaced.

pub fn disabled() -> Self

An inert runtime (metering off) — the default carried when [llm] is absent.

pub fn has_price_book(&self) -> bool

Whether a price book is configured at all. block on an unpriced model only bites when true — a metering-only deployment (no [llm.models]) must not reject every request.

pub fn is_priced(&self, model: &str) -> bool

Whether model carries a price (exact entry or a provider-prefixed alias of one).

pub fn reject_unpriced(&self, model: &str) -> bool

Whether this request must be rejected 402 for an unpriced model: policy is block, a price book exists, and model is not in it. A metering-only setup (empty book) never rejects.

pub fn cost_micros(&self, model: &str, usage: &Usage) -> Option<u64>

Cost of usage for model in micro-dollars, or None if the model has no price (the caller still counts the tokens; whether to serve the request is governed by Self::reject_unpriced). Cached prompt tokens and reasoning completion tokens are billed at their own rate (each defaulting to the base input/output rate), and the remaining prompt/completion tokens at the base rate — so the four dimensions never double-count.

Trait Implementations

impl Clone for LlmRuntime

fn clone(&self) -> LlmRuntime

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LlmRuntime

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.