Skip to main content

Module estimate

Module estimate 

Source
Expand description

The conservative cost estimate a runner checks against a batch’s max_cost_usd before each provider call, and the money arithmetic it needs (RULES.md §5).

Text tokens are bounded by UTF-8 byte length: a tokenizer that works on bytes never makes more tokens than there are bytes, so the bound never underestimates. A module that has the provider’s tokenizer may pass exact counts instead (Tokens::Exact). The fixed costs a role cannot see (the module’s prompt template, the tokens an image costs) are required inputs, never defaulted to zero. Money is integer nanodollars, the catalog’s own unit (RateNanosPerMtok), with checked arithmetic: an overflow refuses, it never wraps.

Structs§

Estimate
An item’s estimate: an upper bound on its input cost.
ItemCostInput
What one item’s provider call is estimated from.

Enums§

EstimateError
Why an item could not be estimated.
Rounding
Which way usd_to_nanos rounds a value finer than a nanodollar.
Tokens
A token count for one part of a request.

Constants§

NANOS_PER_USD
Nanodollars in one dollar.
TOKENS_PER_MTOK
Tokens in the unit a rate is priced per (one million).

Functions§

cost_nanos
The cost of tokens input tokens at rate nanodollars per million, rounded up to a whole nanodollar.
crosses_ceiling
Whether an item estimated at estimate would cross ceiling_nanos after spent_nanos already recorded for the batch. A sum that overflows crosses any ceiling.
estimate_item
Estimate one item’s input cost (RULES.md §5):
nanos_to_usd
Whole nanodollars as a JSON number of USD, for a refusal’s estimate_usd: the exact decimal text, read as a JSON number.
nanos_to_usd_text
Whole nanodollars as an exact decimal number of USD, for a message: 0.00110396, never an exponent form.
usd_to_nanos
A JSON number of USD as whole nanodollars, by decimal string scaling of the number’s text (never through floating-point arithmetic). None for a negative number or one too large for u64 nanodollars.

Type Aliases§

RateNanosPerMtok
A price in integer nanodollars per million tokens. It is the same type as the model catalog’s rate (cortexkit-model-catalog’s RateNanosPerMtok, an i64), so a catalog price passes in as it is, without this crate depending on the catalog.