Skip to main content

Crate cortex_context

Crate cortex_context 

Source
Expand description

§cortex-context

Deterministic, budget-bounded evidence selection for LLM context packets.

You hand it typed evidence items with explicit priorities and trust states plus a token budget. It returns one packet with stable citation IDs, an explicit list of what was omitted, token-savings accounting, and a fail-closed escalation flag — without asking a model to decide what matters.

§Why

Context assembly is usually an implicit, unauditable step: something concatenates strings until a limit is hit. This crate makes it a decision you can inspect and test:

  • Priority, not vibes. Selection order is contradictory-first, then explicit priority, then optional semantic relevance, then submission order.
  • Critical evidence never disappears quietly. If a Critical item does not fit the budget, you get ContextError::CriticalItemExceedsBudget, not a silently shorter prompt.
  • Omissions are reported. omitted_ids tells the caller exactly what was dropped, so a downstream agent can ask for it.
  • Unverified input forces escalation. Any non-verified item sets requires_upstream.
  • Overlap is paid for once. Evidence assembled from several tools repeats itself, and no single tool can see that — each budgets its own answer in isolation. With deduplicate (on by default) a line that a higher-priority item already carried is replaced by same source span as [id] only when span, content, snapshot, blob, trust, and derivation all match. Short lines are never touched, and an item that would render empty keeps its content.
  • Trust is visible in the packet text. Headings carry EXACT SOURCE, UNVERIFIED PLAN, or CONTRADICTORY — group C7, not just an envelope flag.
  • IDs are revision-stable. packetId is pk_<hash>, citations are ev_<hash>, and snapshotId plus per-item blobHash detect a stale packet after the file change.
  • Coverage is a certificate. CoverageCertificate lists required facets, the citations that close each one, missing facets, and contradictions. A boolean sufficient is derived from that ledger.
use cortex_context::{
    compile_context, ContextRequest, EvidenceItem, EvidencePriority, EvidenceState,
};

let request = ContextRequest {
    items: vec![
        EvidenceItem::new(
            "TASK",
            "request:user",
            "Rename the export helper without breaking callers.",
            EvidencePriority::Critical,
            EvidenceState::Verified,
        ),
        EvidenceItem::new(
            "SRC-1",
            "src/export.rs:120",
            "fn render_frontmatter(..) { /* ... */ }",
            EvidencePriority::High,
            EvidenceState::Verified,
        ),
    ],
    max_tokens: 4_000,
    deduplicate: true,
};

let packet = compile_context(&request).expect("the budget fits both items");
assert_eq!(packet.included_ids, ["TASK", "SRC-1"]);
assert!(!packet.requires_upstream);
assert_eq!(packet.deduplicated_lines, 0, "nothing overlapped here");

§Token budgets

compile_context uses a conservative counter so Cyrillic, CJK, and punctuation cannot sneak a packet past the remaining context window. estimate_tokens and CharDiv4Counter stay as the comparative four-character unit for benches. A TokenBreakdown splits budgetOmittedTokens from dedupSavedTokens; those are different kinds of economy and must not be added together as “saved”.

Vendor tokenizers implement TokenCounter. Until one is wired for the active model, runtime compile keeps the conservative fallback.

§Ranking

The ranking module holds pure, deterministic retrieval primitives with pinned parameters: cosine similarity, BM25, reciprocal-rank fusion, and a rank-space structural graph boost. They produce the optional relevance scores above. Nothing here calls a model — embedding vectors come from the caller — and relevance only reorders items within a priority band, so policy always dominates semantics.

§License

Licensed under either of Apache License, Version 2.0 or MIT license at your option.

Modules§

ranking
Deterministic retrieval rankers: cosine similarity, lexical BM25, reciprocal-rank fusion, and structural graph boosting.

Structs§

CharDiv4Counter
Historical comparative unit: chars.div_ceil(4).
ChatMessage
One chat turn for TokenCounter::count_message.
ConservativeCounter
Conservative overestimate for runtime budgets.
ContextPacket
ContextRequest
ContradictionGroup
CoverageCertificate
EvidenceItem
EvidenceLocator
Revision-stable locator. Empty fields mean the gatherer did not know.
FacetClaim
One validated facet close: not “a Dependents fragment existed”.
TokenBreakdown

Enums§

ContextError
EvidenceDerivation
How a fragment was derived. Shown in the packet heading so a model cannot treat an unverified plan as exact source.
EvidenceFacet
Why this fragment is in the packet. Criticality attaches to the facet, not to the Weavatrix operation that produced it.
EvidencePriority
EvidenceState

Constants§

FACET_CALLERS
FACET_CONFIG
FACET_DEFAULTS
FACET_DEFINITION
Facet names the packet can require or expand.
FACET_ERRORS
FACET_EXPANSION_ORDER
Highest-risk missing facet is expanded first.
FACET_GIT
FACET_GUARDS
FACET_MEMORY
FACET_PUBLIC_API
FACET_SIGNATURES
FACET_TESTS
MAX_EVIDENCE_CHARS
MAX_EVIDENCE_ITEMS
MIN_DEDUPLICATED_LINE_CHARS
Lines shorter than this are never deduplicated: a brace, a blank line, or } repeats everywhere and removing it would corrupt an excerpt without saving anything worth having.
RUNTIME_COUNTER

Traits§

TokenCounter
Counts tokens for a budget. Implementations name themselves so a ledger can say which counter produced a number.

Functions§

blob_id
Span body hash: blob_<12 hex>.
compile_context
Select verified evidence by explicit priority without asking a model to decide what matters.
compile_context_with
Same compile, with an explicit counter. Benches that need the historical four-character unit pass CharDiv4Counter.
distrust_caller_verified
Wire-facing compile: a caller cannot mint Verified.
estimate_tokens
Comparative helper used by benches. Runtime compile uses ConservativeCounter.
evidence_id
Content-addressed citation: ev_<12 hex>.
is_critical_facet
packet_id
Packet handle: pk_<12 hex>.
render_decision_map
L0 decision map: intent, snapshot, and a facet ledger.
render_expansions
L2 expansion handles for facets the packet did not close.
snapshot_is_stale
true when the packet was compiled against a different tree than current. A missing packet snapshot cannot prove staleness.