Module autoload_cache

Expand description

autoload_cache submodule. rkyv-backed bytecode cache for autoload functions.

Single-file shard at ~/.zshrs/autoloads.rkyv, keyed by function name. zshrs-original — no C counterpart, so the design rule here is correctness first: an entry that cannot be PROVEN to describe the body about to be installed is a miss.

Storage layout (rkyv archived): AutoloadShard { header: { magic, format_version, zshrs_version, pointer_width, built_at_secs }, entries: HashMap<function_name, AutoloadEntry>, }

Inner chunk_blob is bincode-encoded fusevm::Chunk (same constraint as crate::script_cache module — fusevm::Chunk is upstream and only derives serde).

§What identifies an entry

Three things have to match before a chunk may be run, because a chunk is a function of all three:

the producing binary, as (mtime, len) of current_exe(). Bytecode is not a stable interchange format: the builtin index table and the opcode lowering both move between builds, so a chunk emitted by a different zshrs is meaningless here even when zshrs_version matches. It routinely does match — a debug build and the installed release build share 0.12.36 — which is why the version string cannot carry this check.
the resolved fpath directory, because the same function name lives in several directories on a real $fpath and the cache is keyed by name alone.
the exact definition text, as a SHA-256 of the very string that will be compiled. Not a stat of <dir>/<name>: the body the loader installs can come out of a dir.zwc digest instead of that file, and stamping a path that is not the source is how a chunk built from one text gets served for another.

The previous scheme stamped (mtime, len) of <dir>/<name> and treated the binary check as one-directional (cached < current = stale), so a chunk written by a NEWER binary was served to an older one. On the corpus this was built for that meant _megacomplete being “installed” from a chunk that never defined it, and every <TAB> producing _megacomplete: function not defined by file and zero matches.

Bulk-write: compinit prewarms 16k+ autoload bytecodes in one go. Per-batch shard rewrites (the SQLite-era pattern) would re-serialize 16k entries 160 times. Instead put_many accumulates all entries in memory and writes the shard once. The single-add put_one path remains for the cold-start case where one autoload at a time is compiled by the interactive shell.

The on-disk shape mirrors ScriptShard — same header, same magic-version-pointer_width discipline, same atomic-rename writes.

Structs§

ArchivedAutoloadEntry: An archived AutoloadEntry
ArchivedAutoloadShard: An archived AutoloadShard
ArchivedFileStamp: An archived FileStamp
ArchivedResolvedLoad: An archived ResolvedLoad
ArchivedShardHeader: An archived ShardHeader
AutoloadCache: AutoloadCache — see fields for layout.
AutoloadEntry: AutoloadEntry — see fields for layout.
AutoloadEntryResolver: The resolver for an archived AutoloadEntry
AutoloadShard: AutoloadShard — see fields for layout.
AutoloadShardResolver: The resolver for an archived AutoloadShard
FileStamp: One file’s identity at resolution time.
FileStampResolver: The resolver for an archived FileStamp
MmappedShard: MmappedShard — see fields for layout.
ResolvedLoad: The outcome of resolving and reading one autoload, as loadautofn and the registration step computed it: the body getfpfunc led to and the definition text that body registers as.
ResolvedLoadResolver: The resolver for an archived ResolvedLoad
ShardHeader: ShardHeader — see fields for layout.
ShardHeaderResolver: The resolver for an archived ShardHeader

Constants§

SHARD_FORMAT_VERSION: SHARD_FORMAT_VERSION constant.
SHARD_MAGIC: “ZRAL” little-endian.

Statics§

CACHE: CACHE static.

Functions§

cache_enabled: cache_enabled — see implementation. Honors the process-local script_cache::CACHE_DISABLED AtomicBool first so parity-mode init can disable caches without exporting ZSHRS_CACHE=0 (which would otherwise leak into ${(k)parameters}).
cached_names: cached_names — see implementation.
clear: clear — see implementation.
commit_search: The registration step turned the staged body into registered; record the whole resolution for the entry that compile is about to write.
default_cache_path: default_cache_path — see implementation.
entry_count: entry_count — see implementation.
install_atexit_flush: Register [atexit_flush_pending]. Idempotent; call once from main.
note_resolved: Remember resolved for name; it is written with the next write of that name’s entry (put_one / put_many).
record_port_probe: Record what a native-completer probe found: name lives in dir and its file reads body. Stored as a resolution with no registration and no chunk, which try_port_body and try_source_dir serve and the loader ignores.
source_digest: SHA-256 of the definition text a chunk was compiled from.
stage_hit: loadautofn installed resolved.body from the shard.
stage_search: loadautofn searched $fpath and read body out of dir.
stamp_candidates: The identity of every file getfpfunc could have read for name in dir: the plain file, the per-function dump and the directory digest (c:Src/parse.c:3725 try_dump_file tries <dir>.zwc and <dir>/<name>.zwc before <dir>/<name>).
stats: stats — see implementation.
take_hit: The stored registration for a body loadautofn took from the shard: (registered, from_wordcode, parse_failed). None unless body and the ksh-style decision are the ones it was registered under.
try_flush_pending: Write out everything put_one buffered. See AutoloadCache::flush_pending. Called from preprompt() (the batch boundary for an interactive shell — every autoload a command or a <TAB> triggered lands in one write) and from zexit (the boundary for a script, which never reaches a prompt).
try_load: Raw presence probe for dbview autoloads <name>. See AutoloadCache::get.
try_load_for_source: Execution-path lookup: the compiled definition program for name, valid only for this exact directory and definition text.
try_port_body: The directory and file text for a native completer name, read from the shard: what a getfpfunc existence probe plus a read of <dir>/<name> would return. None for a .zwc-resolved entry (its body is a deparse, not the file) and for anything stale.
try_put_many: Bulk write-through for the prewarm path. See AutoloadCache::put_many.
try_remove: Drop a proven-wrong entry. See AutoloadCache::remove.
try_resolve: The directory and resolved body for name, read from the shard instead of searching $fpath and reading the file.
try_save_one: Write-through after a real autoload compile.
try_source_dir: The $fpath directory the entry for name was resolved from, while that directory is still on fpath and still holds the file. The shard-backed answer to a getfpfunc existence probe (test_only), with the same caveat as try_resolve about a later shadowing file.