Module block_prefetch 

Background block-prefetch worker (RFC 0008 §6).

Periodically snapshots the MetadataIndex, scores each entry per recency / chain-head / model-affinity heuristic, and selects the top-K candidates that the request hot path is most likely to hit next. Goal: warm the flat tier before requests arrive, so cold-S3 load latency disappears from the user-visible TTFT.

Scoring (RFC 0008 §6)

  score = w_recency * exp(-decay * (now - last_access_ns))
        + w_chain   * is_chain_head_bonus
        + w_model   * model_affinity_bonus

With:

• w_recency = 1.0 (primary signal; recently-touched blocks rank highest)
• w_chain = 0.3 (chain-head blocks anchor multi-turn prompts)
• w_model = 0.2 (active model’s blocks rank ahead of stragglers)
• decay = ln(2) / 600e9 (half-life ≈ 10 minutes in nanoseconds)
• is_chain_head_bonus = 1.0 iff BlockMeta.block_seq == 0
• model_affinity_bonus = 1.0 iff BlockMeta.model_digest == active

v1 vs v2

v1 was log-only: scored the candidates and emitted a [MyelonInstr] event listing the top-K, but never issued the GET. v2 (this module) actually fetches: per cycle it issues WombatKVKvStore::get_kv for each top-K miss, materializing the payload into the local flat cache so the next request hits the warm path.

The fallback path is preserved behind the WMBT_KV_PREFETCH_DRY_RUN=1 env: when set, the worker scores and logs but never issues GETs (matches v1 behavior for diagnostic / canary deployments).

Sequential vs parallel

v2 issues GETs sequentially per cycle. This mirrors the C ABI’s per-block path (Handle::put_kv_blocks parallelizes via std::thread::scope, but get_kv itself is the cabi’s per-block call). Parallel fetch within a cycle is a v3 TODO, once we have evidence of cycle-time becoming a bottleneck, swap in a std::thread::scope fan-out bounded by top_k.

Structs

PrefetchConfig

Tunables for the prefetch worker.

PrefetchFetchOutcome

Per-cycle outcome counts. Surfaced via the [MyelonInstr] event emitted by default_v2_emit.

PrefetchPlan

One cycle’s prefetch plan. Held briefly inside the worker thread, then handed to emit (v1 / dry-run) or to the fetcher (v2).

PrefetchWorker

Owns a background thread that scores + would-prefetch hot blocks.

Traits

PrefetchFetcher

Materialization surface for v2 prefetch. The worker holds an Arc<dyn PrefetchFetcher> so the algorithm crate can issue GETs without the block_prefetch module depending on the embed module’s generic WombatKVKvStore<S> shape.

Functions

block_key_for_hash

Compose the relative block key for a BlockHash. Mirrors wombatkv-cabi::block_key_for_hash, both call sites read the same wombatkv_radix::BLOCK_KEY_PREFIX so they can never skew.

default_emit

Default v1 / dry-run emit: a [MyelonInstr] JSON line on stderr per cycle. Mirrors the existing event shape in embed.rs so log parsers see one consistent envelope across read/write/prefetch paths.

default_v2_emit

Default v2 emit: a [MyelonInstr] JSON line per cycle with full stage counts (scored, selected, skipped_already_flat, fetched, failed, bytes_materialized, elapsed_ms).

dry_run_enabled

Returns true if the v1 / dry-run fallback is requested via env. WMBT_KV_PREFETCH_DRY_RUN=1 (and the usual truthy synonyms) flips the embed-side start_prefetcher back to log-only behavior.

run_cycle

Run one scoring + selection cycle against the supplied index. Returns the plan; callers decide what to do with it (emit, GET, etc.).

run_cycle_v2

Run one v2 cycle: score, select, fetch top-K, return the per-stage counts. Exposed for tests so the cycle can run inline (without thread orchestration).

score_block

Score BlockMeta under the RFC 0008 §6 heuristic.

spawn_worker

Spawn the v1 / dry-run worker. Scores + logs, never fetches.

spawn_worker_v2

Spawn the v2 worker that actually fetches the top-K each cycle.

Type Aliases

PrefetchEmit

Closure-based callback for “I would have prefetched this block”. Lets tests assert the worker’s choices without coupling to the log format.