Module context 

The deterministic context compiler (EPIC-P-070): classify, dedup, and pack caller-supplied context fragments under a token budget — no LLM, no cloud, every decision auditable. Gated behind the default context feature. The deterministic context compiler (EPIC-P-070).

Classifies, deduplicates, and packs caller-supplied context fragments under a token budget — no LLM, no network, no clock: the pipeline is a sequence of pure stages (chunk → classify → dedup → score → pack → assemble), so the same CompileRequest always produces the same CompiledContext, byte for byte.

Invariants:

• Budget: the assembled content never exceeds the request’s token budget — packing accounts per-piece estimates plus joiner costs priced by the injected estimator, which bounds the whole-text estimate for a superadditive estimator (the default rounds every piece up).
• Provenance: every input fragment gets exactly one ContextDecision with a stable rule id and a content hash; every fragment stays addressable via a content-addressed ctx://source/<hash> handle (immune to caller-id collisions) — hashed over the text for a text fragment, over the raw decoded media bytes for a media fragment (see Analysis::handle_hash: captions are typically blank, so a caption-keyed handle would collide every captionless image).
• Nothing critical is silently lost: content that cannot fit becomes a RetrievalHandle; losing preserve-classified content raises CompiledContext::risk to FidelityRisk::High; a critical fragment is never sacrificed to near-deduplication, and a duplicate of a twin that did not emit verbatim keeps its own handle and risk.

Memory-backed fragment selection, persisted working contexts, and compilation events layer on top in the persistence-gated bridge (US-002); MCP and Node expose the same types unchanged (US-003).

Re-exports

pub use chunk::chunk_text;

pub use chunk::ChunkBoundary;

pub use chunk::ChunkPolicy;

pub use chunk::TextChunk;

pub use estimator::DynTokenEstimator;

pub use estimator::HeuristicEstimator;

pub use estimator::TokenEstimator;

pub use ingest::IngestRoots;

pub use insights::CompilationInsights;

pub use insights::ModelPricing;

pub use insights::PricingTable;

pub use model::CompilePolicy;

pub use model::CompileRequest;

pub use model::CompiledContext;

pub use model::CompiledSection;

pub use model::ContextAction;

pub use model::ContextDecision;

pub use model::ContextDecisionRef;

pub use model::ContextFact;

pub use model::ContextFragment;

pub use model::ContextSavings;

pub use model::ContextSource;

pub use model::ContextWarning;

pub use model::FidelityRisk;

pub use model::ImportanceWeights;

pub use model::LoadedWorkingContext;

pub use model::MediaRef;

pub use model::MemoryScope;

pub use model::RetrievalHandle;

pub use model::SectionKind;

pub use model::SourceReference;

pub use model::WorkingContext;

pub use model::WorkingContextIndex;

pub use model::WorkingContextSession;

pub use model_windows::model_window;

pub use model_windows::suggest_token_budget;

pub use model_windows::SuggestedBudget;

pub use segment::segment_transcript;

pub use segment::SegmentFormat;

pub use segment::SegmentKind;

pub use segment::SegmentationOutcome;

pub use segment::SegmentationPolicy;

pub use segment::TranscriptSegment;

pub use transcript_bridge::build_transcript_compile_request;

pub use transcript_bridge::SegmentInfo;

pub use transcript_bridge::SegmentationReport;

pub use transcript_bridge::TranscriptCompileInput;

Modules

chunk

Deterministic text chunking for the packing stage.

estimator

Pluggable token estimation, with a deterministic char-class default.

ingest

Adapter-side I/O pre-pass for path-referenced context fragments (V2b-1): resolves ContextFragment::path into content under a strict, short-circuiting security pipeline, BEFORE the request reaches the pure compiler core. Not compiled for wasm32 — there is no local filesystem to read from a WASM host; see crate::error::MemoryError::IngestDisabled for what a path fragment does there instead. Adapter-side I/O pre-pass for path-referenced context fragments (V2b-1, see the crate’s PLAN.md).

insights

Savings accounting: tokens (always) and money (only when a pricing table is injected).

model

Data model of the context compiler: the request/response value types.

model_windows

The suggest_budget MCP tool’s static model→window table (V2a-3 quick win). No dependency on anything else in the pipeline — a pure lookup. A static, committed model → context-window table (V2a-3 quick win).

segment

Deterministic transcript segmentation for the compile_transcript MCP tool (V2b-2): splits a raw agent-session transcript into turns and, within each turn, into code/log/body sub-segments — pure, zero-regex, zero-clock, so the same transcript + policy always segments byte-identically. See segment::segment_transcript. Deterministic transcript segmentation for the compile_transcript MCP tool (V2b-2, see the crate’s PLAN.md, section V2b).

transcript_bridge

The binding-side glue over segment: one implementation of “transcript in, CompileRequest + audit trail out”, relayed by the Node, Python and WASM bindings instead of copied into each. The one place a binding turns a raw transcript into a CompileRequest plus an auditable segmentation report.

wire

The id wire contract (decimal-string u64) shared by every JS-facing binding of these types — one source of truth for wire::ID_KEYS instead of a Node/WASM copy each. JSON-tree helpers for the id wire contract shared by every JS-facing binding (Node, WASM) of the context types: a u64 id crosses as a decimal string, because JS number loses precision above 2^53.

Structs

ContextCompiler

The deterministic context compiler. Build one with a policy, optionally inject an estimator and a pricing table, then compile.

DeterministicReranker

The first shipped Reranker: re-orders a fused candidate pool by deterministic lexical overlap with the query, original (fused) order as the tie-break. Never invents or drops ids, never calls a model — safe to wire into recall_fused_reranked where a cross-encoder would be overkill or non-reproducible.

Functions

fragment_id

The stable, content-addressed id of a fragment whose caller supplied none — the crate’s one id scheme (FNV-1a 64), also used as every decision’s content hash and as the tail of every ctx://source/<hash> handle.