Module cache 

Content-addressed cache for LLM responses.

Two design choices diverge from the Python drep/llm/cache.py and the criterion is “the cache should hit on the call paths the gate actually takes”, not “the cache should mirror what 1.x did”:

• The key is content-only, never commit-aware. The Python mixed the commit SHA into the key so a new commit invalidated everything. Hashing the content already invalidates precisely when the content changes; the SHA was forcing a miss on every unchanged file at every new commit, which is the exact case the cache exists to serve. On a pre-commit gate that is the common path.
• One file per entry, the value is the JSON. No sidecar metadata file. The key is the filename, so nothing has to be re-validated on read except age. If the entry is corrupt, unreadable, or expired, the read returns None and the caller re-queries and overwrites. A cache is an optimisation; it must not be able to take the gate down.

Storage layout

root/<first two hex chars>/<full hex>.json. Sharding keeps directories from growing to tens of thousands of entries (a flat root/<hex>.json layout would do the same for readdir, but ls-ing the cache to debug it would take seconds).

Key composition

blake3 over the six inputs, each length-prefixed with an 8-byte big-endian length. Length prefixing rules out the key("ab", "c", ...) vs key("a", "bc", ...) collision that a separator byte cannot guarantee once content or system_prompt is allowed to contain that byte. temperature is formatted with a fixed number of decimals so 0.2 and 0.20 hash to the same key (they are the same value).

The backend identity is part of the key, not just the model. A model name is not a globally unique identity: the canonical failover pair is one open model served from a local runtime and from a cloud provider, which name it identically. Keyed on the model alone, the fallback’s answer lands where the head would look for its own - so a later run with the head restored is served a response it never produced, which is the exact defect the per-provider key exists to prevent.

Defaults

30 days TTL, 1 GiB max bytes.

Structs

Cache

The cache: a directory tree of one JSON file per entry, keyed by blake3 digest of the six prompt and backend inputs.

CacheKey

A cache key: the blake3 hex digest of the six key inputs.

Enums

CacheError

What can go wrong at write time.