poly-kv

Shared compressed KV-cache pool. One pool, many agents, zero leaks.

poly-kv is the pool primitive for multi-agent inference. It solves a specific problem: 10 agents share the same 200-token system prompt, but without compression, you're storing that prompt 10 times in VRAM. With poly-kv, you store it once, compress it 50×, and give each agent a 17ms shell for its unique tokens.

Compressed-domain scoring scores attention candidates from compressed codebook indices without decompressing to f32, then decodes only the selected top-k values. On CPU, multi-head batch scoring is 4.98x faster than optimized exact f32 attention at 16K tokens, and 5.81x faster at 32K (AMD Ryzen 7 7730U, release mode). Quality is near-lossless at candidate_k=48 (+0.67% PPL on DistilGPT2/WikiText-103).

This is the production target of the governed compression workspace: a typed, receipted, two-tier pool where the shared tier is built once and read by everyone.

The two-tier strategy

KV-cache compression is hard because one size doesn't fit all. Shared context (system prompts, few-shot examples, retrieval results) needs high compression — it's large, rarely changes, and you can afford some fidelity loss. Agent-private context (conversation turns, tool outputs) needs near-lossless reconstruction — it's smaller but critical for correctness.

poly-kv gives you both:

Tier	What it holds	Codec	Compression	Fidelity	Build cost
Shared pool (cold)	System prompts, shared context	fib-quant k=4, N=32	~50× theoretical	cos 0.863	1,557ms (once)
Agent shell (hot)	Per-agent conversation, tool results	turbo-quant 8-bit, 32proj	~8× theoretical	cos 0.9996	17ms (per agent)

The shared pool is read-only after build — agents physically cannot contaminate each other. Every operation produces a typed receipt that captures what was built, who read it, and what was returned.

What's in the box

`SharedKVPool` (`src/pool.rs`, 602 lines)

The shared, immutable, build-once KV-cache pool. Built from a corpus of vectors; encoded with fib-quant; addresses documents by their content digest.

use poly_kv::SharedKVPool;

let pool = SharedKVPool::build(corpus, profile)?;
// 1,557ms for 80 shared docs, 768-dim, fib-quant k=4.
// Receipt: PoolBuildReceipt { profile_digest, block_digest, ... }

The build is deterministic — same corpus + same profile

same seed = same block_digest. You can verify the pool matches a previous build by comparing digests.

`AgentShell` (`src/shell.rs`, 324 lines)

A per-agent overlay that materializes the agent's private context on top of the shared pool. Built fast (17ms for 12 docs); uses turbo-quant for the hot tier (cosine 0.9996 fidelity).

use poly_kv::AgentShell;

let shell = AgentShell::materialize(&pool, agent_docs, profile)?;
// 17ms for 12 docs, 768-dim, turbo-quant 8-bit.
// Receipt: ShellMaterializeReceipt { shell_digest, ... }

Manifests (`src/manifest.rs`, 179 lines)

A typed PoolManifest that names the corpus, the profile, the codec, the seed, the block layout, the receipt, and the content digest. The manifest is what gets serialized to disk and what gets verified when you re-open the pool.

Receipts (`src/receipt.rs`, 373 lines)

Every operation produces a typed receipt:

PoolBuildReceipt — emitted on SharedKVPool::build. Carries the profile, the seed, the block digest, the per-block statistics.
ShellMaterializeReceipt — emitted on AgentShell::materialize. Carries the shell digest, the agent's read set, the cost.
FallbackReceiptV1 — emitted when an exact-fallback was triggered. Carries the reason and the original path.

All three receipts are BLAKE3-hashed and signed with the codec profile digest, so the audit trail is tamper-evident.

Policy (`src/policy.rs`, 209 lines)

A typed policy object that the caller passes to build and materialize. The policy is the single decision point: "this corpus is admissible for fib-quant with these parameters, and the result is admissible for these admissibility classes."

Exact fallback (`src/fallback.rs`, 66 lines)

The contract: any compressed representation can be re-derived back to its raw input. If a caller asks for Admissibility::Exact and the codec can't deliver, the adapter falls back to raw and emits a FallbackReceiptV1.

Compressed candidate read path

poly-kv now has two compressed-domain attention selection APIs:

SharedKVPool::attention_topk_compressed(...) scores cold-pool Fib codes, selects top-k tokens, and decodes only selected values.
AgentShell::attention_topk_compressed(...) scores both the Fib cold pool and the Turbo hot shell, performs one global top-k, and decodes only selected values.

The shell path is the proveKV bridge:

query
  -> compressed Fib cold-pool key scores
  -> compressed Turbo hot-shell key scores
  -> global top-k
  -> decode selected values only
  -> CompressedAttentionSelectionReceipt

Receipt fields include candidate/source counts, selected pool/shell counts, decoded value count, full_layer_decoded=false, and an explicit claim boundary. This is compressed candidate-selection evidence only; model-quality/KV-cache preservation claims require exact attention and logit/PPL replay receipts.

Model-shaped replay gate

poly-kv also includes a deterministic model-shaped replay harness:

cargo run --example poly_kv_model_replay_receipt \
  > docs/codex-runs/P3/POLY_KV_MODEL_REPLAY_RECEIPT.json

The replay compares AgentShell::attention_topk_compressed(...) against an exact full-decode attention reference, then projects both attention outputs through a deterministic synthetic output head. The receipt records attention-output cosine/MSE, projected-logit KL, top-1 agreement, PPL-proxy delta, candidate-k sweep results, decoded value count, and full-decode value count.

Stored receipt:

docs/codex-runs/P3/POLY_KV_MODEL_REPLAY_RECEIPT.json
docs/codex-runs/P3/POLY_KV_MODEL_REPLAY_SUMMARY.md

Current stored result: candidate_k=32 passed the local synthetic replay gate with output cosine 0.9981, KL 0.00109, top-1 agreement 1.0, and 18.0x fewer decoded value vectors than full decode.

Claim boundary: this is deterministic model-shaped replay over a synthetic projection. It is not real model PPL, production KV-cache preservation, production latency evidence, or provider/framework KV-cache byte-reduction evidence. The next gate is captured Q/K/V/logit replay from a small local model.

Captured-tensor replay gate

poly-kv now has a captured-tensor replay API and fixture path:

python3 tools/capture_tiny_transformer_replay.py \
  --out docs/codex-runs/P3/POLY_KV_CAPTURED_TINY_TRANSFORMER_FIXTURE.json

cargo run --example poly_kv_captured_model_replay -- \
  docs/codex-runs/P3/POLY_KV_CAPTURED_TINY_TRANSFORMER_FIXTURE.json \
  > docs/codex-runs/P3/POLY_KV_CAPTURED_MODEL_REPLAY_RECEIPT.json

The capture script emits Q/K/V rows, exact attention output, output projection, exact logits, and labels from a deterministic NumPy tiny-transformer forward pass. The Rust replay API rebuilds the Fib cold pool and Turbo hot shell from those captured tensors, runs compressed candidate selection, and compares compressed outputs against the captured exact tensors/logits.

Stored receipt:

docs/codex-runs/P3/POLY_KV_CAPTURED_TINY_TRANSFORMER_FIXTURE.json
docs/codex-runs/P3/POLY_KV_CAPTURED_MODEL_REPLAY_RECEIPT.json
docs/codex-runs/P3/POLY_KV_CAPTURED_MODEL_REPLAY_SUMMARY.md

Current stored result: selected candidate_k=16, output cosine 0.5847, KL 0.00622, top-1 agreement 0.25, PPL-proxy delta -4.6746, and 4.5x fewer decoded value vectors than full decode.

Claim boundary: this is captured-tensor fixture evidence from a deterministic NumPy tiny transformer, not pretrained LLM PPL preservation. torch and transformers were not installed on the host, and Ollama does not expose Q/K/V tensors/logits via its normal API.

Pretrained DistilGPT2 captured replay gate

poly-kv now includes a dependency-light pretrained capture path for distilgpt2:

# one-time tiny dependency env used by the capture script
python3 -m venv .venv-capture
.venv-capture/bin/pip install safetensors tokenizers sentencepiece huggingface_hub numpy

HF_HUB_DISABLE_XET=1 .venv-capture/bin/python tools/capture_distilgpt2_replay.py \
  --out docs/codex-runs/P3/POLY_KV_CAPTURED_DISTILGPT2_FIXTURE.json

cargo run --example poly_kv_captured_model_replay -- \
  docs/codex-runs/P3/POLY_KV_CAPTURED_DISTILGPT2_FIXTURE.json \
  > docs/codex-runs/P3/POLY_KV_CAPTURED_DISTILGPT2_RECEIPT.json

Stored artifacts:

docs/codex-runs/P3/POLY_KV_CAPTURED_DISTILGPT2_FIXTURE.json
docs/codex-runs/P3/POLY_KV_CAPTURED_DISTILGPT2_RECEIPT.json
docs/codex-runs/P3/POLY_KV_CAPTURED_DISTILGPT2_SUMMARY.md

The capture script loads pretrained distilgpt2 safetensors and tokenizer files, runs a manual NumPy forward pass, captures layer-0/head-0 Q/K/V rows and exact model logits, and emits a replay fixture for the Rust gate. This closes the "real pretrained captured tensors" setup gap without requiring torch/transformers.

Current stored result: diagnostic negative. The strict captured replay gate fails at candidate_k=72 with output cosine 0.6512, KL 6.2431, top-1 agreement 0.0, PPL-proxy delta 573.3435, and no decode reduction at the selected fallback budget. This says the current single-head projection proxy is not sufficient evidence for KV-cache preservation.

Claim boundary: this is pretrained DistilGPT2 captured-tensor evidence and a negative diagnostic receipt. It is not full-forward model-quality preservation, real corpus PPL preservation, production KV-cache preservation, production latency evidence, or a replacement for KIVI/KVQuant/Quest.

Pretrained DistilGPT2 full-forward intervention gate

poly-kv now has the next replay gate: compressed candidate attention is reinjected into the downstream manual DistilGPT2 forward path before comparing final logits.

.venv-capture/bin/python tools/distilgpt2_full_forward_intervention.py \
  --out docs/codex-runs/P3/POLY_KV_DISTILGPT2_FULL_FORWARD_INTERVENTION_RECEIPT.json \
  --summary docs/codex-runs/P3/POLY_KV_DISTILGPT2_FULL_FORWARD_INTERVENTION_SUMMARY.md

Stored artifacts:

docs/codex-runs/P3/POLY_KV_DISTILGPT2_FULL_FORWARD_INTERVENTION_RECEIPT.json
docs/codex-runs/P3/POLY_KV_DISTILGPT2_FULL_FORWARD_INTERVENTION_SUMMARY.md

Current stored result: pass at candidate_k=8 on the deterministic DistilGPT2 prompt/window. The compressed sparse intervention decodes 548 selected value vectors vs 2,628 full-decode value vectors (4.7956x decode reduction), with final-logit KL 0.000832, top-1 agreement 1.0, attention-output cosine 0.8916, and PPL-proxy delta -0.0062.

Claim boundary: this is a pretrained DistilGPT2 full-forward intervention receipt for one deterministic prompt/window, layer 0, head 0, with a per-vector int8 compressed candidate selector. It is not real-corpus PPL preservation, production KV-cache preservation, production latency evidence, or a replacement for KIVI/KVQuant/Quest.

Pretrained DistilGPT2 held-out full-forward suite

The next coverage gate runs the same full-forward intervention over three fixed held-out prompts and two heads:

.venv-capture/bin/python tools/distilgpt2_full_forward_suite.py \
  --out docs/codex-runs/P3/POLY_KV_DISTILGPT2_FULL_FORWARD_SUITE_RECEIPT.json \
  --summary docs/codex-runs/P3/POLY_KV_DISTILGPT2_FULL_FORWARD_SUITE_SUMMARY.md

Stored artifacts:

docs/codex-runs/P3/POLY_KV_DISTILGPT2_FULL_FORWARD_SUITE_RECEIPT.json
docs/codex-runs/P3/POLY_KV_DISTILGPT2_FULL_FORWARD_SUITE_SUMMARY.md

Current stored result: pass across 6 prompt/head cases. Aggregate pass rate 1.0, final-logit KL mean 0.000980 / max 0.003947, final top-1 agreement mean/min 1.0, attention-output cosine mean 0.9649 / min 0.9264, abs PPL-proxy delta mean 0.01395 / max 0.04752, decode-reduction mean/min 4.2975x.

Pretrained DistilGPT2 layer/head coverage suites

The broader coverage gate reuses the full-forward intervention suite with explicit layer/head coverage:

.venv-capture/bin/python tools/distilgpt2_full_forward_suite.py \
  --suite-label all-head \
  --layer 0 \
  --heads 0,1,2,3,4,5,6,7,8,9,10,11 \
  --candidate-ks 8 \
  --out docs/codex-runs/P3/POLY_KV_DISTILGPT2_LAYER0_ALL_HEADS_SUITE_RECEIPT.json \
  --summary docs/codex-runs/P3/POLY_KV_DISTILGPT2_LAYER0_ALL_HEADS_SUITE_SUMMARY.md

.venv-capture/bin/python tools/distilgpt2_full_forward_suite.py \
  --suite-label layer-sweep \
  --layers 0,1,2,3,4,5 \
  --heads 0 \
  --candidate-ks 8 \
  --out docs/codex-runs/P3/POLY_KV_DISTILGPT2_LAYER_SWEEP_SUITE_RECEIPT.json \
  --summary docs/codex-runs/P3/POLY_KV_DISTILGPT2_LAYER_SWEEP_SUITE_SUMMARY.md

Stored artifacts:

docs/codex-runs/P3/POLY_KV_DISTILGPT2_LAYER0_ALL_HEADS_SUITE_RECEIPT.json
docs/codex-runs/P3/POLY_KV_DISTILGPT2_LAYER0_ALL_HEADS_SUITE_SUMMARY.md
docs/codex-runs/P3/POLY_KV_DISTILGPT2_LAYER_SWEEP_SUITE_RECEIPT.json
docs/codex-runs/P3/POLY_KV_DISTILGPT2_LAYER_SWEEP_SUITE_SUMMARY.md

Layer-0 all-head result: pass across 36 prompt/head cases, final-logit KL mean 0.02151 / max 0.33674, final top-1 agreement mean 0.9792 / min 0.5, attention-output cosine mean 0.9168 / min 0.5972, abs PPL-proxy delta mean 0.09590 / max 1.6993, decode-reduction mean/min 4.2975x.

Layer-sweep result: pass across 18 prompt/layer cases over layers 0-5 head 0, final-logit KL mean 0.000661 / max 0.003947, final top-1 agreement mean/min 1.0, attention-output cosine mean 0.9707 / min 0.9264, abs PPL-proxy delta mean 0.01140 / max 0.04752, decode-reduction mean/min 4.2975x.

Claim boundary: these are broader DistilGPT2 full-forward intervention coverage receipts over fixed prompts, layer 0 all heads, and all layers for head 0. They are stronger than the 2-head suite but still not real-corpus PPL preservation, production KV-cache preservation, production latency evidence, simultaneous multi-head/multi-layer intervention, or a replacement for KIVI/KVQuant/Quest.

Isolated attention speed benchmark

The speed benchmark isolates the operation being optimized: exact dense single-head attention over precomputed DistilGPT2 Q/K/V tensors vs the current compressed sparse candidate path. It excludes setup/full-forward cost and records timing separately from decode-work reduction.

.venv-capture/bin/python tools/distilgpt2_attention_speed_bench.py \
  --seq-len 512 \
  --layers 0 \
  --heads 0,1,2,3,4,5,6,7,8,9,10,11 \
  --candidate-k 8 \
  --warmup 5 \
  --repeat 30 \
  --out docs/codex-runs/P3/POLY_KV_DISTILGPT2_ATTENTION_SPEED_BENCH_RECEIPT.json \
  --summary docs/codex-runs/P3/POLY_KV_DISTILGPT2_ATTENTION_SPEED_BENCH_SUMMARY.md

Stored artifacts:

docs/codex-runs/P3/POLY_KV_DISTILGPT2_ATTENTION_SPEED_BENCH_RECEIPT.json
docs/codex-runs/P3/POLY_KV_DISTILGPT2_ATTENTION_SPEED_BENCH_SUMMARY.md

Current stored result: batch-optimized path is the fastest compressed variant. It shows 20.59x fewer selected value decodes. After excluding setup and quality diagnostics from the timed hot path, batch-optimized compressed timing achieves speed_ratio_exact_over_batch=0.5822x (exact_time / compressed_time) with some cases exceeding 1.0x. Optimized/prequantized is 0.4585x. Quest-style page-filtered is 0.1752x (slower due to Python page-scan overhead). Scalar diagnostic is 0.1502x and vectorized-with-setup is 0.1977x. This proves decode-work reduction and shows that batch position processing is the highest-ROI Python-level optimization, but the aggregate still does not prove runtime speedup.

Claim boundary: isolated NumPy CPU attention-operator benchmark over precomputed DistilGPT2 Q/K/V tensors. Setup/full-forward cost is excluded, and the optimized/batch/quest timing excludes quality diagnostics from timed hot paths. The Quest-style page min/max pre-filter is based on arXiv:2406.10774. It is not production runtime speedup, not GPU/kernel evidence, and not end-to-end generation latency evidence.

Claim boundary: this is held-out DistilGPT2 full-forward intervention evidence over a small fixed prompt/head suite. It is stronger than one-prompt replay but still not real-corpus PPL preservation, production KV-cache preservation, production latency evidence, all-layer/all-head validity, or a replacement for KIVI/KVQuant/Quest.

Quick Start

use poly_kv::{SharedKVPool, AgentShell, Admissibility};
use quant_codec_core::CodecProfile;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // 1. Build the shared pool (cold tier).
    let shared_corpus: Vec<Vec<f32>> = /* system prompts + shared docs */;
    let shared_pool = SharedKVPool::build(shared_corpus, profile.clone())?;
    println!("Shared pool built: {:?}", shared_pool.build_receipt);

    // 2. Each agent materializes a shell (hot tier) on top.
    for agent_id in 0..10 {
        let agent_docs: Vec<Vec<f32>> = /* agent-private context */;
        let shell = AgentShell::materialize(&shared_pool, agent_docs, profile.clone())?;
        println!("Agent {} shell: {:?}", agent_id, shell.materialize_receipt);
    }
    Ok(())
}

Run it: cargo run --release --example poly_kv_fast_roundtrip.

Benchmarks — measured

10-agent contention (June 2026)

10 agents, 80 shared docs (768-dim, fib-quant k=4), 12 agent-private docs per agent (768-dim, turbo-quant 8-bit):

Metric	Result
Agents with recall@1 = 1.0	10/10
Cross-agent top-1 leaks	0/90 pairs
Pool build (80 shared docs)	1,557ms
Shell materialize (12 docs/agent)	17ms avg
fib-quant cold compression batch	480 KB → 133 KB (3.6× JSON, ~48× binary projected)
turbo-quant hot fidelity	cosine 0.9996

Every agent found its target at rank 1. Zero interference. The shared pool is read-only after build — agents physically cannot contaminate each other.

Single-route parity (June 2026)

8 queries, 200 docs, 768-dim, k=10:

Route	Recall@1	Recall@10	nDCG@10	Rank drift
exact_scan (no compression)	1.000	1.000	1.000	—
fib-quant only	1.000	1.000	1.000	0.33
turbo-quant only	1.000	1.000	1.000	0.03
poly-kv (two-tier)	1.000	1.000	1.000	0.25

"Do All" perf pass (2026-06-01) — pool build throughput

After the June 1 perf pass (AVX2+FMA SIMD + Rayon parallel across vec_idx + Rayon parallel across layer_idx):

Config	qwen3 n=4	qwen3 n=20	qwen3 n=80	nomic n=4	nomic n=20	nomic n=80
Old (f64 reference)	1449ms	4271ms	13763ms	459ms	1336ms	4552ms
+ SIMD	418ms	—	—	—	—	94ms
+ Rayon (parallel)	893ms	968ms	1250ms	271ms	296ms	407ms
+ parallel_pool (full)	256ms	291ms	346ms	94ms	100ms	133ms

Best speedup over old (f64): 5.7× at qwen3 n=4, 40× at qwen3 n=80.

The 28 layers in qwen3 are independent — spreading them across cores compounds the fib-quant Rayon wins.

GPU path (msi i7-6700HQ + GTX 1070)

Shape	n	wall CPU	wall Hadamard-GPU	wall Hadamard+Codebook-GPU
nomic 768	80	4552	4430	4485
qwen3 2560	80	13763	13419	13428

Hadamard-only win: 2.5-2.7% on the larger corpora. Hadamard + Codebook-GPU win: 1.5-2.4%. The new codebook kernel is slower in integration than just the Hadamard alone because per-call H2D/D2H transfer overhead dominates.

The kernel is correct (parity test passes for n=32, d=128, k=4, N=32 random inputs on msi GTX 1070). The dispatch is the issue, not the kernel.

JSON vs binary storage

Current compression ratios are JSON-serialized. The JSON envelope is 12× bigger than the actual codebook indices. Binary wire format is the next PR. PackedTurboCode already exists in turbo-quant. PackedFibCode is next.

	JSON (current)	Binary (projected)
Shared pool (80 docs)	240 KB → 66 KB (3.6×)	240 KB → ~5 KB (48×)
Agent shell (12 docs)	36 KB → 63 KB (0.6×)	36 KB → ~5 KB (7×)
System total (200 docs)	600 KB → 695 KB (0.9×)	600 KB → ~95 KB (6.3×)

Test coverage

4 integration test files in tests/:
- integration_tests.rs (162 lines) — full build-then-materialize roundtrip with receipts.
- pool_tests.rs (118 lines) — pool invariants: determinism, immutability, profile digest stability.
- receipt_tests.rs (169 lines) — receipt roundtrip, BLAKE3 digest stability, fallback contract.
- shell_tests.rs (151 lines) — agent shell contracts: materialization, isolation, cost.
4 examples in examples/:
- poly_kv_dynamic_cache_roundtrip.rs — full end-to-end build + materialize + search.
- poly_kv_fast_roundtrip.rs — fast path benchmark.
- poly_kv_gpu_bench.rs — GPU dispatch benchmark.
- test_compact_decode.rs — compact binary decode.
1 bench in benches/synthetic_pool.rs.
25+ Python validation scripts in scripts/ for preflight, schema validation, public claim checking, receipt integrity, source-package hygiene, and final-state validation.
cargo test clean, cargo clippy --all-targets -- -D warnings clean.

MSRV

Rust 1.75 (2021 edition). Stable features only.

Benchmarks

Speed — compressed vs exact attention (CPU)

Real-kernel comparison on AMD Ryzen 7 7730U (8 cores, release mode). Baseline: optimized cache-blocked exact f32 attention with partial top-k heap sort.

Multi-head batch scoring (the production-relevant path):

tokens	head_dim	heads	ratio (compressed/opt exact)
2,048	64	12	1.14x
4,096	64	12	1.55x
8,192	64	12	2.54x
16,384	64	12	4.98x
32,768	64	12	5.81x
4,096	128	8	1.22x
8,192	128	8	2.47x
16,384	128	8	3.54x

The win region is 4K–32K tokens — exactly where multi-agent serving workloads live.

Quality — real-corpus PPL (DistilGPT2)

WikiText-103 sample, all 6 layers × 12 heads compressed, top-k candidate selection:

candidate_k	delta PPL	KL divergence	top-1 agreement
8	+71.77%	0.825	0.54
16	+34.90%	0.541	0.56
32	+7.15%	0.178	0.77
48	+0.67%	0.040	0.87
64	-0.13%	0.010	0.95

Break-even at k=48 (near-lossless), effectively lossless at k=64.

Claim boundaries

Speed: synthetic random vectors, isolated attention operator, CPU only, not end-to-end latency
Quality: DistilGPT2 only (6 layers, 12 heads), NumPy CPU forward, exact f32 scoring (not compressed Gram estimates), single WikiText-103 sample, seq_len=128
Not production serving, not GPU, not large model evidence
All receipts stored as typed JSON in docs/codex-runs/P3/

Dependencies

serde (with derive).
serde_json.
blake3.
rand + rand_chacha.
thiserror.
turbo-quant (optional) — for the turbo feature.
fib-quant (optional) — for the fib feature.
gpu-backend (optional) — for the GPU path.
rayon (optional) — for the parallel pool build.

License

MIT OR Apache-2.0 (dual-licensed). See LICENSE-MIT and LICENSE-APACHE for the full texts.

Changelog

See CHANGELOG.md for the release history.

Scope and limits

This crate is alpha. The following claims are explicitly forbidden in documentation, rustdoc, README, and release notes unless scoped to a specific external paper claim or local receipt evidence (per the AGENTS.md release-claim law):

"zero accuracy loss"
"zero overhead"
"production KV cache runtime"
"drop-in replacement"
"better than semantic-memory"
"proven deployment quality"
"no dataset-specific calibration needed"

What's allowed:

"experimental pool primitive"
"two-tier codec policy (fib-quant cold + turbo-quant hot)"
"receipt-bearing, deterministic, runnable on synthetic fixtures"
"exact-fallback contract enforced"
"scope: shared KV-cache pool, not full agent runtime"

Attribution

This crate is an independent Rust implementation of the PolyKV-style shared compressed KV-cache pool idea. It is not the original authors' reference implementation and does not claim affiliation with the PolyKV paper authors. The pool architecture, the two-tier strategy, the receipt infrastructure, the GPU dispatch path, and the test suite are original to this implementation.

Where it's used

poly-kv is the pool primitive for:

The LLM runtime stack — when 10+ agents share a system prompt, the shared pool saves N× memory and gives each agent a 17ms spin-up cost.
semantic-memory — when the corpus has a stable shared component (system prompts, few-shot examples), the shared pool reduces the per-import cost.
The quant-governor policy layer — when the policy routes a multi-agent workload to the cold tier, the SharedKVPool is what actually serves the request.

Any system that needs shared, immutable, receipted vector storage across multiple consumers can adopt poly-kv directly.

poly-kv 0.1.0-alpha.3

poly-kv

The two-tier strategy

What's in the box

SharedKVPool (src/pool.rs, 602 lines)

AgentShell (src/shell.rs, 324 lines)

Manifests (src/manifest.rs, 179 lines)

Receipts (src/receipt.rs, 373 lines)

Policy (src/policy.rs, 209 lines)

Exact fallback (src/fallback.rs, 66 lines)

Compressed candidate read path

Model-shaped replay gate

Captured-tensor replay gate

Pretrained DistilGPT2 captured replay gate

Pretrained DistilGPT2 full-forward intervention gate

Pretrained DistilGPT2 held-out full-forward suite

Pretrained DistilGPT2 layer/head coverage suites

Isolated attention speed benchmark

Quick Start

Benchmarks — measured

10-agent contention (June 2026)

Single-route parity (June 2026)

"Do All" perf pass (2026-06-01) — pool build throughput

GPU path (msi i7-6700HQ + GTX 1070)

JSON vs binary storage

Test coverage

MSRV

Benchmarks

Speed — compressed vs exact attention (CPU)

Quality — real-corpus PPL (DistilGPT2)

Claim boundaries

Dependencies

License

Changelog

Scope and limits

Attribution

Where it's used

`SharedKVPool` (`src/pool.rs`, 602 lines)

`AgentShell` (`src/shell.rs`, 324 lines)

Manifests (`src/manifest.rs`, 179 lines)

Receipts (`src/receipt.rs`, 373 lines)

Policy (`src/policy.rs`, 209 lines)

Exact fallback (`src/fallback.rs`, 66 lines)