pub struct TextIndexSignal {
pub present: bool,
pub bytes: u64,
pub token_table_bytes: Option<u64>,
}Expand description
The file’s full-text-search affordance: does it carry a TEXT_INDEX section (kind 6), and what does that index cost?
A .rete built with --text-index answers FILTER(CONTAINS(…)) by word
lookup; one built without it answers the same query with the same rows by
full scan. The capability is invisible from the results, which is exactly how
the playground catalog came to advertise an index two published files never
carried (#189) — so a file has to be able to state it.
Measured at read time, never stored. The ground truth is the section
directory in the 1 KiB header that every card read already fetches, so a
stored copy would be a second source of truth for a fact the first source
answers for free — the same reasoning that keeps a curated language field
out of the card. Being a projection rather than a stamp also means every
already-published file reports it today, with no re-card, and that
rete repyramid --text-index (which rewrites the file’s sections) can never
leave it stale.
What it deliberately does not carry: a token count. The number of
distinct indexed words is the first varint of the decompressed token
table, so quoting it would mean fetching and inflating the whole table —
193 MB on the published causenet-full-typed.rete. token_table_bytes is
the same question answered for ≤10 bytes, and is the more useful answer
anyway: it is what a first search actually costs.
Fields§
§present: boolTrue iff the file has a kind-6 section. Always written, including as
false: “measured, and there is none” is the fact #189 was missing.
bytes: u64Byte length of the whole TEXT_INDEX section — free, straight from the
header. Worth stating because it can dominate the download:
causenet-full-typed.rete carries 1.88 GB of it, 29% of the file.
token_table_bytes: Option<u64>Byte length of the section’s leading token table — the prefix a
first search faults, several times smaller than bytes (which counts
the postings blob, fetched one posting at a time). One ≤10-byte range
read; None when there is no index, or its first bytes were unreadable.
Implementations§
Source§impl TextIndexSignal
impl TextIndexSignal
Sourcepub fn probe<R>(reader: &R, header: &Header) -> TextIndexSignalwhere
R: RangeReader + ?Sized,
pub fn probe<R>(reader: &R, header: &Header) -> TextIndexSignalwhere
R: RangeReader + ?Sized,
Measure the signal from a file’s header, through the same
crate::RangeReader the card was read with. Costs nothing beyond
the header already in hand when there is no index, and one ≤10-byte
range read when there is — never the index itself.
Trait Implementations§
Source§impl Clone for TextIndexSignal
impl Clone for TextIndexSignal
Source§fn clone(&self) -> TextIndexSignal
fn clone(&self) -> TextIndexSignal
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read more