Expand description
Budget middleware and shipped BudgetHandle implementations (plan §10).
The BudgetHandle trait itself lives in oharness-core; this crate
provides the concrete implementations (TokenBudget, StepBudget,
CostBudget, TimeBudget, CompositeBudget), a PricingTable for
cost calculations, and BudgetMiddleware — an Llm wrapper that
pre-checks, post-consumes on complete(), and observes Chunk::Usage
events on stream().
BudgetMiddleware implements oharness_llm::Llm directly (the escape
hatch per plan §5.6.2) rather than via a helper-trait wrapper, because it
needs to thread the same budget counter through all three hook sites.
Feature flags per plan §16.1: token and step are on by default;
cost and wall-clock opt-in.
Re-exports§
pub use amount::amount_from_response;pub use amount::amount_from_usage;pub use amount::budget_request_from;pub use composite::CompositeBudget;pub use error::BudgetExceeded;pub use middleware::BudgetMiddleware;pub use pricing::ModelPricing;pub use pricing::PricingTable;pub use step::StepBudget;pub use token::TokenBudget;
Modules§
- amount
- Helpers to project
CompletionRequest/CompletionResponse/ per-chunkUsagevalues onto theBudgetAmount/BudgetRequestshapes that theoharness_core::BudgetHandleAPI expects. - composite
CompositeBudget— fanscheck/consume/snapshotacross any number of innerBudgetHandles. Any single child returningDenydenies the whole composite (plan §10.2).- error
- The sentinel error returned when a budget is exhausted.
- middleware
BudgetMiddleware— anLlmwrapper that threads a sharedBudgetHandlecounter through three hook sites:- pricing
- Per-model pricing table used by
CostBudgetand the cost component ofBudgetAmount(plan §10.4). - step
StepBudget— hard cap on the number of completed LLM calls (plan §10.2).- token
TokenBudget— hard cap on the sum of input + output tokens consumed over the lifetime of the handle (plan §10.2).