Skip to main content

Module model_limits

Module model_limits 

Source
Expand description

Each driver choice’s per-turn output cap, context window, input ceiling and input budget, resolved once at launch.

A transport: http capsule names a model and the driver that serves it. The driver’s bundled murmur.yaml declares, in its models: table, each model’s max_output_tokens, context_window and optionally max_input_tokens (read by murmur_artifact::parse_model_limits). For every driver choice, the primary and each inference.alternates entry:

Limit"manifest"else "driver"else
output capinference.max_tokensthe table’s max_output_tokensrefused, E-RUN-052
context windowcontext.max_tokensthe table’s context_windownone, "none"
input ceiling—the table’s max_input_tokensnone, "none"

The manifest’s two keys are session-wide: each applies to every choice as written, never clamped to a declared ceiling. A model id matches a table entry by exact string equality. A choice whose model the table does not list draws W-RUN-009 unless the manifest sets both keys.

Every inference call has one input budget, min(input ceiling, context window − output cap) with the cap the call sends (ResolvedModelLimits::input_budget_for). Compaction’s threshold and the seed budget are fractions of it. With no window there is no budget: for that choice compaction does not run and no seed budget applies. A choice whose budget is 0 is refused, E-RUN-055.

inference.compaction.max_tokens is checked here too: above the primary driver’s ceiling for the compaction model it is refused, E-RUN-054; at or above the compaction model’s window it leaves compaction calls no input budget, E-RUN-055.

Staging and mur doctor both resolve through resolve_model_limits, so the two never disagree on a number or on the warning’s text. transport: process resolves nothing: the harness owns its model and its output.

Structs§

ChoiceModelLimits
One driver choice’s resolution.
DriverManifest
A driver choice’s driver, as resolution reads it.
InputBudget
The most input tokens one inference call may carry, and which limit set it.
ResolvedModelLimits
One driver choice’s output cap, context window, input ceiling and input budget in force, and where each came from. The model_limits object of session_start and of each inference_choices element.

Enums§

BudgetBasis
Which term of min(input ceiling, context window − output cap) an input budget is.
LimitSource
Where a limit in force came from, as session_start names it.
ModelLimitsError
Why a launch is refused: E-RUN-052, E-RUN-054 for ModelLimitsError::CompactionCapAboveCeiling, or E-RUN-055 for ModelLimitsError::NoInputBudget and ModelLimitsError::CompactionNoInputBudget.

Functions§

limit_or_none
n, or none for an absent limit, as the refusals, mur doctor and MURMUR.md print it.
model_limits_warning_line
W-RUN-009 as staging and mur doctor print it, message being ChoiceModelLimits::warning.
resolve_model_limits
Resolves the output cap, context window, input ceiling and input budget of every driver choice inference declares, the primary first and the alternates in manifest order. Empty under transport: process.