Module cache

Expand description

Caller-owned stability annotations, provider-owned placement mechanics.

Structs§

CachePolicy: The caller’s x-cache annotation: stability boundaries plus an optional cache identity. Read once per request and never forwarded to a provider.
CacheSegment: One caller-declared stability boundary: every message up to and including upto_message is expected to stay byte-stable for as long as stability says.

Stability: How long the caller expects a prefix to stay byte-stable. Only the caller knows; llmshim never infers it.

continuation_identity: Settings identity for a stateless continuation. History is supplied in full each time; include/store/reasoning and all other settings remain in the hash. This does not cache or store any conversation on the provider.
continuation_matches: Compare settings and the complete prior message prefix. This does not enable provider-side storage or replace the next request with a delta.
finish_request: Final native-body pass, after provider overrides. Never forward x-cache.
prepare_request: Annotate the original message indices before system extraction or native tool-message conversion. Without x-cache this is an exact clone/no-op.
uses_extended_ttl