Module bytecode_cache

Expand description

Content-addressed on-disk cache for compiled .harn pipelines.

Cold-start harn run re-parses, type-checks, and compiles the entry pipeline before the VM gets a single instruction to execute. For short Harn subcommands that wrap a few llm_calls in a small pipeline, that compile cost dominates wall-clock time.

This module persists Chunk bytecode under $HARN_CACHE_DIR/<source-hash>.harnbc (XDG-aware). The cache key is derived from the entry source plus the content hash of every transitively-imported user file; stdlib imports are covered by the embedded harn_version field in the header. Any change to any input flips the key and the next run recompiles.

File layout — little-endian throughout:

magic        : [u8; 8]   = "HARNBC\0\0"
schema_ver   : u32       = SCHEMA_VERSION
version_len  : u32
harn_version : [u8; version_len]
compiler_tag : u8        bitmask of active CompilerOptions
kind         : u8        1 = entry chunk, 2 = module artifact
source_hash  : [u8; 32]
import_hash  : [u8; 32]
payload      : bincode-serialized payload for `kind`

The header lets a stale binary detect a future-version artifact without crashing: a magic mismatch, schema mismatch, or version mismatch is returned as Ok(None) so the caller transparently recompiles. Real I/O errors propagate.

Concurrency: writes are atomic (write-tmp-then-rename), and parallel invocations on a cache miss race safely — the last writer wins, but every reader observes a consistent file because the rename is atomic on every supported filesystem.

Structs§

CacheKey: Cache key components for a single pipeline source. Equality of all fields is necessary and sufficient for cache reuse.
LookupOutcome: Result of a cache lookup. Carries the precomputed key so the caller can write it back on a miss without rehashing.
ModuleLookupOutcome: Result of a load_module lookup. Carries the precomputed key so the caller can write it back on a miss without rehashing.

Constants§

CACHE_DIR_ENV: Environment override for the cache directory. When set, takes precedence over the XDG and home-directory fallbacks.
CACHE_ENABLED_ENV: Environment override that turns the cache off entirely. Setting this to 0, false, no, or off skips both reads and writes; useful when debugging compiler changes.
CACHE_EXTENSION: Conventional extension for entry-chunk cache files.
HARN_VERSION: Compile-time Harn release. Cache files written by a different release are rejected on load.
MAGIC: Header magic for all bytecode-cache artifact families.
MODULE_CACHE_EXTENSION: Conventional extension for module-artifact cache files. Distinct from CACHE_EXTENSION so the same .harn source can have both shipped adjacent if needed (e.g. when a file is both an executable entry and imported by other files).
SCHEMA_VERSION: On-disk format version. Bump when CachedChunk or the header layout changes in a backwards-incompatible way.

Functions§

adjacent_cache_path: Path to the adjacent precompiled entry-chunk artifact for source_path. foo.harn → foo.harnbc.
adjacent_module_cache_path: Path to the adjacent precompiled module-artifact for source_path. foo.harn → foo.harnmod.
cache_dir: Returns the directory the shared cache lives in. Honors $HARN_CACHE_DIR, then $XDG_CACHE_HOME/harn/bytecode, then $HOME/.cache/harn/bytecode. The directory is not created here — store creates it lazily on write so read-only environments don’t pay an mkdir cost.
cache_enabled: True when the cache is enabled by the current environment.
load: Try to load a cached chunk for source_path whose contents are source. Returns the key alongside the (optional) chunk so callers avoid recomputing the key on miss.
load_module: Look up the ModuleArtifact for source_path (whose contents are source). Mirrors load but for the .harnmod family.
packs_cache_dir: Root for .harnpack archives unpacked by harn run <bundle.harnpack>. Each verified bundle is replayed into <root>/<sanitized-bundle-hash>/ so re-runs reuse the unpacked tree. Honors $HARN_CACHE_DIR/packs when set, otherwise XDG / $HOME/.cache/harn/packs.
serialize_chunk_artifact: Serialize an entry-chunk artifact (header + payload) to bytes. The resulting buffer is byte-identical to the file store_at would have written for the same (key, chunk). Use this when packaging artifacts into a container (e.g. harn pack) without going through the filesystem.
serialize_module_artifact: Serialize a module artifact (header + payload) to bytes. Companion to serialize_chunk_artifact for the .harnmod family.
store: Persist chunk to the shared cache directory under key. Atomic: a temp file is written then renamed into place. Concurrent invocations on the same key race safely.
store_at: Write a precompiled entry-chunk artifact to an explicit path, for use by the harn precompile subcommand. The header still records the key, so adjacent artifacts shipped with source are validated like any other cache hit.
store_module: Persist artifact to the shared cache under key. Atomic; concurrent invocations race safely.
store_module_at: Write a module artifact to an explicit path.