Expand description
Normative identifier extraction for compaction retention (INV-C2).
Anchored-iterative compaction replaces the conversation’s anchor summary wholesale, so an identifier the summary drops is dropped forever. The retention invariant (INV-C2, conformance row CONF-2) is stated over extractable identifiers — and this module is the single, normative definition of that term. The compaction recall eval (#1134) and the mint-time retention gate (#1135) both call this extractor; neither may ship a private variant, or the eval would measure a different property than the gate enforces.
§Token classes
An extractable identifier is a token (or token run) in one of five classes, matching the conformance suite’s ambiguity resolution:
IdentifierClass::Url— anhttp://orhttps://token, trailing punctuation trimmed.IdentifierClass::Amount— a numeral with a currency symbol ($/€/£), a trailing%, a following unit word fromUNIT_WORDS, or at least four digits (order numbers, PINs).IdentifierClass::OpaqueId— a machine-shaped token: at least five characters drawn from[A-Za-z0-9._/-]containing both a letter and a digit (UUIDs,ord_93k2f7x,ZK-4471-BQ).IdentifierClass::ProperNoun— a run of two or more consecutive capitalized words, leading English function words stripped (Mirela Okafor, notThe Mirela).IdentifierClass::Quoted— the content of a straight double-quoted span of 3–120 bytes containing a letter. This is how an open question stays trackable: prose that tags it (open item: "night berthing at dock 7") makes the tag extractable, while free interrogative prose is semantic content the recall eval measures but the gate cannot extract.
§Survival predicate
An identifier survives into a candidate text when the candidate
contains its text verbatim (case-sensitive substring —
is_retained). A paraphrase (Okafor, Mirela) does not count: the
gate’s failure mode on over-strictness is rejecting a candidate summary
and keeping the prior anchor, which is the safe direction (INV-C3).
Structs§
- Identifier
- One extracted identifier: its class and its verbatim text.
Enums§
- Identifier
Class - The class of an extracted identifier. See the module docs for the normative grammar of each class.
Constants§
- UNIT_
WORDS - Unit words that promote a bare numeral to an
IdentifierClass::Amount.
Functions§
- extract_
identifiers - Extract every identifier in
text, deduplicated and in a stable (class, text) order. - is_
retained - The normative survival predicate:
candidateretainsidentifierwhen it contains the identifier’s text verbatim (case-sensitive substring). - missing_
identifiers - The identifiers extracted from
priorthat do NOT survive (peris_retained) intocandidate.