Expand description
§cortex-context
Deterministic, budget-bounded evidence selection for LLM context packets.
You hand it typed evidence items with explicit priorities and trust states plus a token budget. It returns one packet with stable citation IDs, an explicit list of what was omitted, token-savings accounting, and a fail-closed escalation flag — without asking a model to decide what matters.
§Why
Context assembly is usually an implicit, unauditable step: something concatenates strings until a limit is hit. This crate makes it a decision you can inspect and test:
- Priority, not vibes. Selection order is
contradictory-first, then explicit priority, then optional semantic relevance, then submission order. - Critical evidence never disappears quietly. If a
Criticalitem does not fit the budget, you getContextError::CriticalItemExceedsBudget, not a silently shorter prompt. - Omissions are reported.
omitted_idstells the caller exactly what was dropped, so a downstream agent can ask for it. - Unverified input forces escalation. Any non-verified item sets
requires_upstream. - Overlap is paid for once. Evidence assembled from several tools repeats
itself, and no single tool can see that — each budgets its own answer in
isolation. With
deduplicate(on by default) a line that a higher-priority item already carried is replaced bysame source span as [id]only when span, content, snapshot, blob, trust, and derivation all match. Short lines are never touched, and an item that would render empty keeps its content. - Trust is visible in the packet text. Headings carry
EXACT SOURCE,UNVERIFIED PLAN, orCONTRADICTORY — group C7, not just an envelope flag. - IDs are revision-stable.
packetIdispk_<hash>, citations areev_<hash>, andsnapshotIdplus per-itemblobHashdetect a stale packet after the file change. - Coverage is a certificate.
CoverageCertificatelists required facets, the citations that close each one, missing facets, and contradictions. A booleansufficientis derived from that ledger.
use cortex_context::{
compile_context, ContextRequest, EvidenceItem, EvidencePriority, EvidenceState,
};
let request = ContextRequest {
items: vec![
EvidenceItem::new(
"TASK",
"request:user",
"Rename the export helper without breaking callers.",
EvidencePriority::Critical,
EvidenceState::Verified,
),
EvidenceItem::new(
"SRC-1",
"src/export.rs:120",
"fn render_frontmatter(..) { /* ... */ }",
EvidencePriority::High,
EvidenceState::Verified,
),
],
max_tokens: 4_000,
deduplicate: true,
};
let packet = compile_context(&request).expect("the budget fits both items");
assert_eq!(packet.included_ids, ["TASK", "SRC-1"]);
assert!(!packet.requires_upstream);
assert_eq!(packet.deduplicated_lines, 0, "nothing overlapped here");§Token budgets
compile_context uses a conservative counter so Cyrillic, CJK, and
punctuation cannot sneak a packet past the remaining context window.
estimate_tokens and CharDiv4Counter stay as the comparative
four-character unit for benches. A TokenBreakdown splits
budgetOmittedTokens from dedupSavedTokens; those are different
kinds of economy and must not be added together as “saved”.
Vendor tokenizers implement TokenCounter. Until one is wired for
the active model, runtime compile keeps the conservative fallback.
§Ranking
The ranking module holds pure, deterministic retrieval primitives with
pinned parameters: cosine similarity, BM25, reciprocal-rank fusion, and a
rank-space structural graph boost. They produce the optional relevance
scores above. Nothing here calls a model — embedding vectors come from the
caller — and relevance only reorders items within a priority band, so
policy always dominates semantics.
§License
Licensed under either of Apache License, Version 2.0 or MIT license at your option.
Modules§
- ranking
- Deterministic retrieval rankers: cosine similarity, lexical BM25, reciprocal-rank fusion, and structural graph boosting.
Structs§
- Char
Div4 Counter - Historical comparative unit:
chars.div_ceil(4). - Chat
Message - One chat turn for
TokenCounter::count_message. - Conservative
Counter - Conservative overestimate for runtime budgets.
- Context
Packet - Context
Request - Contradiction
Group - Coverage
Certificate - Evidence
Item - Evidence
Locator - Revision-stable locator. Empty fields mean the gatherer did not know.
- Facet
Claim - One validated facet close: not “a Dependents fragment existed”.
- Token
Breakdown
Enums§
- Context
Error - Evidence
Derivation - How a fragment was derived. Shown in the packet heading so a model cannot treat an unverified plan as exact source.
- Evidence
Facet - Why this fragment is in the packet. Criticality attaches to the facet, not to the Weavatrix operation that produced it.
- Evidence
Priority - Evidence
State
Constants§
- FACET_
CALLERS - FACET_
CONFIG - FACET_
DEFAULTS - FACET_
DEFINITION - Facet names the packet can require or expand.
- FACET_
ERRORS - FACET_
EXPANSION_ ORDER - Highest-risk missing facet is expanded first.
- FACET_
GIT - FACET_
GUARDS - FACET_
MEMORY - FACET_
PUBLIC_ API - FACET_
SIGNATURES - FACET_
TESTS - MAX_
EVIDENCE_ CHARS - MAX_
EVIDENCE_ ITEMS - MIN_
DEDUPLICATED_ LINE_ CHARS - Lines shorter than this are never deduplicated: a brace, a blank line, or
}repeats everywhere and removing it would corrupt an excerpt without saving anything worth having. - RUNTIME_
COUNTER
Traits§
- Token
Counter - Counts tokens for a budget. Implementations name themselves so a ledger can say which counter produced a number.
Functions§
- blob_id
- Span body hash:
blob_<12 hex>. - compile_
context - Select verified evidence by explicit priority without asking a model to decide what matters.
- compile_
context_ with - Same compile, with an explicit counter. Benches that need the historical
four-character unit pass
CharDiv4Counter. - distrust_
caller_ verified - Wire-facing compile: a caller cannot mint
Verified. - estimate_
tokens - Comparative helper used by benches. Runtime compile uses
ConservativeCounter. - evidence_
id - Content-addressed citation:
ev_<12 hex>. - is_
critical_ facet - packet_
id - Packet handle:
pk_<12 hex>. - render_
decision_ map - L0 decision map: intent, snapshot, and a facet ledger.
- render_
expansions - L2 expansion handles for facets the packet did not close.
- snapshot_
is_ stale truewhen the packet was compiled against a different tree thancurrent. A missing packet snapshot cannot prove staleness.