Module cache_runtime 

v2.40.0 — the result-memoization cache core.

This is the production-hardened runtime behind the cache primitive. The type checker (v2.40.0) already proved WHAT is safe to cache (a pure tool by construction; a widened one only with a finite TTL); this module implements HOW, with the properties a naïve cache omits and that cause real outages:

• Content-addressed, deploy-safe, tenant-isolated keys: the key is a hash of (tenant ‖ cache ‖ tool ‖ tool-declaration-fingerprint ‖ output_type ‖ selected params). A redeploy that changes a tool changes its fingerprint → a new key → no stale cross-deploy hit; the tenant is a key component → no cross-tenant leak even if a backend mis-namespaces.
• Single-flight: concurrent misses for one key compute ONCE; the rest wait for that result (no thundering herd).
• Provable-forever, never non-deterministic-forever: enforced at compile time; the runtime simply honours the (optional) TTL.
• Production hygiene: errors are never cached; oversized values are not cached (never truncated into a wrong value); TTL expiry is jittered (deterministically, per key) so entries don’t expire in a herd.

The CacheBackend trait lets the enterprise inject a Redis (multi-replica) tier; with none injected, the in-process tier is fully functional single-replica.

Structs

CachePlan

v2.89.0 — everything a deployment memoises, resolved once from the IRProgram.

CacheRuntime

Ties policy resolution + content-addressed key derivation + single-flight compute-through into one call the dispatch path makes per tool. The enterprise injects a Redis backend + the real tenant; the OSS default is an in-process backend under a "local" tenant. This is the whole runtime contract for v2.40.0 — a hit returns before compute runs (so a budget gate placed after the lookup never sees it, the design decision).

CacheSlot

v2.89.0 — a reserved place to put a computed value, handed out by CacheRuntime::probe on a miss and consumed by CacheRuntime::store.

InProcessCache

The OSS default single-replica tier: a bounded map with per-entry TTL (jittered), LRU eviction, a value-size bound, and single-flight miss coalescing via per-key locks.

ResolvedCachePolicy

Everything the dispatch path needs to key ONE memoised call, resolved from the IRProgram once, when the plan is built.

Enums

CacheOutcome

The outcome of a cache-mediated dispatch — lets the caller emit the right cache:hit / cache:miss audit signal without re-deriving it.

CacheProbe

v2.89.0 — what a pre-dispatch probe found.

Constants

DEFAULT_CAPACITY

Default cap on the in-process tier (entries), mirroring IdempotencyStore.

DEFAULT_MAX_VALUE_BYTES

Default per-value size ceiling (bytes). An oversized result is simply not cached — never truncated into a wrong value.

Traits

CacheBackend

A pluggable cache tier. namespace is the cache declaration’s name so invalidate can flush exactly one cache’s entries. The enterprise injects a Redis impl of this; the OSS default is InProcessCache.

Functions

derive_key

Derive the content-addressed cache key. key_args are the selected (param_name, value) pairs (the full bound set, or the key: subset).

parse_duration

Parse a duration literal ("10s", "500ms", "5m", "2h", "1d") to a Duration. None for a malformed string (the lexer already guarantees the shape for a ttl: field, so this is defence in depth).

resolve_invalidation_channels

Channel name → the cache namespaces an emit on it flushes (invalidate_on:).

resolve_tool_cache

Resolve which cache (if any) governs a tool’s memoization, given the whole program IR. Precedence: an explicit cache: none opts out; an explicit cache: <Name> selects that cache; otherwise the single default: true cache applies IFF the tool is eligible (provably pure, or its effects are a subset of the default’s apply_to_effects). Returns None when nothing caches the tool.

resolve_tool_cache_policies

Resolve, for every tool in the program, which cache (if any) memoises it — keyed by TOOL NAME, which is how the dispatch path knows a tool.

tool_fingerprint

The stable fingerprint of a tool’s DECLARATION — a hash of its IR spec. A redeploy that changes the tool’s provider, effects, output type, or parameters changes this, so a behaviour change can never serve a result cached under the old behaviour.