Skip to main content

Module reminder

Module reminder 

Source
Expand description

How full the window is, from the provider’s own count.

Usage is the input_tokens the provider reported for the last call — a fact — against the model’s context_tokens from the endpoint book. Never a byte count of the window, which is an estimate, and never a tokenizer of this repository’s own, which would be a second one.

Two thresholds, and each sounds exactly once per run. At the first rung — 30% of the window — only the usage is reported; at the second rung — 65% of the window unless a layer moved it — the run is told that what is left is still enough to write a handoff and succeed, and that past this point it will not be. A gauge that jumps past both in one reading sounds the higher one only: two sentences stacked at once are noise, and the lower one has nothing left to say.

Structs§

ContextGauge
One run’s gauge. Fed the provider’s input_tokens after every call; answers a reminder the first time each threshold is crossed.
ContextReading
The provider’s count for the last call a run completed, shared with whoever reports it.

Enums§

ContextReminder
The line the run is given, with the figures it is computed from.