Module cache 

Content-addressed cache for LLM responses.

Two choices make cache hits follow the call paths the gate actually takes:

• The key is content-only, never commit-aware. Hashing content invalidates precisely when the content changes; including the commit SHA would force a miss on unchanged files after every commit, which is the exact case the cache exists to serve.
• One file per entry, the value is the JSON. No sidecar metadata file. The key is the filename, so nothing has to be re-validated on read except age. If the entry is corrupt, unreadable, or expired, the read returns None and the caller re-queries and overwrites. A cache is an optimisation; it must not be able to take the gate down.

Storage layout

root/<first two hex chars>/<full hex>.json. Sharding keeps directories from growing to tens of thousands of entries (a flat root/<hex>.json layout would do the same for readdir, but ls-ing the cache to debug it would take seconds).

Key composition

blake3 over the six inputs, each length-prefixed with an 8-byte big-endian length. Length prefixing rules out the key("ab", "c", ...) vs key("a", "bc", ...) collision that a separator byte cannot guarantee once content or system_prompt is allowed to contain that byte. temperature uses Rust’s shortest round-tripping representation, so equal values hash alike without collapsing neighboring f32 values.

The request identity is conservative. It contains the wire protocol, max_tokens, and the effective header set. Header names are lower-cased and sorted before they reach the hash, because spelling does not change an HTTP request; values remain exact because an arbitrary header can select a tenant, model route or feature variant and drep cannot infer that from its name. A token rotation therefore cold-starts that provider’s cache. That cost is the safe default: reusing an answer from a different route is a false claim about which model reviewed the file. Values are hash input only and are never written into the cache entry or its path as text.

The backend identity is part of the key, not just the model. A model name is not a globally unique identity: the canonical failover pair is one open model served from a local runtime and from a cloud provider, which name it identically. Keyed on the model alone, the fallback’s answer lands where the head would look for its own - so a later run with the head restored is served a response it never produced, which is the exact defect the per-provider key exists to prevent.

Defaults

30 days TTL, 256 MiB max bytes.

Structs

Cache

The cache: a directory tree of one JSON file per entry, keyed by blake3 digest of the six prompt and backend inputs.

CacheKey

A cache key: the blake3 hex digest of the six key inputs.

Enums

CacheError

What can go wrong at write time.