Module script_cache

Expand description

script_cache submodule. rkyv-backed bytecode cache for zsh scripts.

Single-file shard at ~/.zshrs/scripts.rkyv. On 2+ runs of a given script, lex/parse/compile is skipped — the cache hit is mmap + zero-copy ArchivedHashMap lookup + bincode-decode of the inner fusevm::Chunk blob.

Storage layout (rkyv archived): ScriptShard { header: { magic, format_version, zshrs_version, pointer_width, built_at_secs }, entries: HashMap<canonical_path, ScriptEntry>, } ScriptEntry { mtime_secs, mtime_nsecs, binary_mtime_at_cache, binary_len_at_cache, cached_at_secs, chunk_blob: Vec<u8> }

Inner chunk_blob is bincode for now — fusevm::Chunk is owned by the upstream fusevm crate and only derives serde::Serialize/Deserialize, not rkyv::Archive, so the inner codec stays bincode inside the rkyv outer container. Direct rkyv on Chunk would require either forking fusevm or a mirror archived type — both are large refactors and not needed for the current “kill SQLite for bytecode” goal.

Read path:

Lazy mmap of the shard, kept alive for the process lifetime so repeat lookups pay validation once.
rkyv::check_archived_root::<ScriptShard> validates the byte image.
Header validated for magic / format_version / zshrs_version / pointer_width.
Per-entry: source mtime must match, and the entry’s recorded binary identity (binary_mtime_at_cache, binary_len_at_cache) must EQUAL the running binary’s (mtime, len). Bytecode is only ever replayed by the exact build that emitted it; any rebuild invalidates entries silently.

Write path:

bin_zsystem_flock(LOCK_EX) on scripts.rkyv.lock so concurrent writers serialize.
Read existing shard into owned form, mutate, rkyv::to_bytes, write to scripts.rkyv.tmp.<pid>.<nanos>, fsync, atomic-rename.
Drop the in-process mmap so the next read picks up the new shard.

Run log: a cache hit never rewrites the shard. Its time is appended to scripts.runs (unix_secs\tpath, flushed with the pending puts) and overlaid on dbview scripts; the next shard write folds it into cached_at_secs and removes the log.

Ported from strykelang/strykelang/script_cache.rs (the user’s stryke language has the same caching pattern; this is the same shape with ZRSC magic, zshrs version pin, and a single chunk_blob per entry — zshrs has no separate AST cache).

Structs§

ArchivedScriptEntry: An archived ScriptEntry
ArchivedScriptShard: An archived ScriptShard
ArchivedShardHeader: An archived ShardHeader
GuardedEvent: One executed event of a recorded file: its compiled chunk, plus what a replay needs to know that the chunk is still the program the lexer would produce now.
MmappedShard: mmap + validated *const ArchivedScriptShard. Self-referential — the pointer is valid for the lifetime of the wrapping struct.
PendingPut: One buffered write: what try_save_bytes queues until the next flush.
ScriptCache: Shard cache keyed by canonical script path. One per shard file.
ScriptEntry: ScriptEntry — see fields for layout.
ScriptEntryResolver: The resolver for an archived ScriptEntry
ScriptShard: ScriptShard — see fields for layout.
ScriptShardResolver: The resolver for an archived ScriptShard
ShardHeader: ShardHeader — see fields for layout.
ShardHeaderResolver: The resolver for an archived ShardHeader

Enums§

BlobKind: What a cached blob holds. A top-level script is one chunk; a sourced file is the sequence of per-event chunks the loop() of c:Src/init.c:155-220 compiled, each lexed under the state the previous ones left behind.

Constants§

SHARD_FORMAT_VERSION: Bumped on incompatible rkyv schema changes.
SHARD_MAGIC: Magic header bytes — fail-fast if a wrong-format file is mmap’d. “ZRSC” little-endian.

Statics§

CACHE: Process-wide ScriptCache rooted at default_cache_path(). None when the cache is disabled or the path could not be opened.
CACHE_DISABLED: Process-local disable flag set by parity-mode flags (--zsh etc.) in bins/zshrs.rs. Preferred over ZSHRS_CACHE=0 in env so the env var doesn’t leak into ${(k)parameters} and inflate the param count vs reference zsh.

Functions§

cache_enabled: ZSHRS_CACHE=0|false|no (env) or CACHE_DISABLED=true (process- local) disables the cache entirely.
capture_begin: Start recording the events of path, which the caller is about to run.
capture_commit: Finish the innermost recording. complete is the caller’s verdict that the file ran to its end or to a deliberate return / exit — not to a parse error or a failed event.
capture_commit_all_on_exit: zexit is committed: every file still being recorded stops here, at the event that called exit, and a replay stops at the same event. Skipped in a forked child (see [Capture::pid]) and when an error flag is live.
capture_push: Append the chunk of the event about to run to the innermost recording. start is the event’s byte offset in the text the loop is reading. Called BEFORE the event runs, so an exit inside it still finds it, and with the alias tables exactly as the event was lexed under.
clear: clear — see implementation.
default_cache_path: Default shard path: $ZSHRS_HOME/scripts.rkyv (default ~/.zshrs/scripts.rkyv).
env_fingerprint: Hash of everything the lexer and compiler read from the live shell: the alias tables, the option store and the emulation mode. Two runs that start from equal fingerprints lex the same bytes into the same chunks.
env_generation: (ALIAS_GEN, option generation) — compared before and after a run to learn whether the run itself changed lexer-visible state.
evict_stale: evict_stale — see implementation.
file_mtime: file_mtime — see implementation.
stats: stats — see implementation.
try_flush_pending: Write out everything try_save_bytes buffered, in one shard rewrite. Called at the prompt, from zexit and from the atexit hook, alongside the autoload and deparse caches. Cheap when nothing is buffered.
try_load_bytes: Try to load a cached program by source path. Returns None on any miss, including a different lexer-state fingerprint or a blob of another kind.
try_save_bytes: Store the bincode-encoded program for a script path. env_fp is the fingerprint taken BEFORE the run: the state the lexer started from, not the state the file left behind. Best-effort — cache disabled / canonicalize failure / mtime stat failure all return Ok(()) silently so the caller can fire-and-forget.