Skip to main content

Module embed

Module embed 

Source
Expand description

Embeddable KV-cache store API for inference engines.

Combines the in-process FoyerHybridCache (G2 RAM + G3 NVMe) with a durable object store (G4: S3, MinIO, local fs via wombatkv_store::wal_store::InMemoryObjectStore for tests) so a single inference engine can:

  1. Write a KV blob through both tiers (foyer hot, S3 cold) on prefill.
  2. Read with foyer-first, S3-fallback semantics on decode / cold start.
  3. Restart cleanly: a fresh process can rehydrate foyer from S3 on boot, so no work is lost when the engine restarts.

The store is generic over wombatkv_store::wal_store::ObjectStore, so the same code path is exercised by unit tests (in-memory) and by live MinIO/S3 integration tests.

Structs§

EmbedConfig
Tuning knobs for the embeddable KV store.
WombatKVKvStore
Embeddable KV cache store.

Enums§

EmbedError
Errors surfaced by the embeddable KV store.
GetOutcome
Outcome of a get_kv call.
HitTier
A KV cache lookup hit, with the tier the value came from.

Functions§

emit_timing