Module range 

Stable byte-range and lazy-open API for local or remote .rete files.

Structs

BlockCacheReader

ByteRange

A byte range in the .rete file image.

CountingReader

Wraps a reader and tallies how many ranges were requested and how many bytes were returned — the metric that matters for a range-streamed format. Atomically counted, so it stays Sync (a lazily-faulting remote index holds its reader behind a shared loader).

LayoutSegment

One labelled byte region of a .rete file image (see Rete::file_layout). kind is a stable machine tag: header, metadata, dictionary, directory, tile, pyramid, named-graphs.

OffsetReader

Wraps a RangeReader so every access is shifted by base bytes: a .rete that starts at byte base of the backing resource — e.g. behind an HTML shell in a polyglot file that is both a web page and a graph — reads exactly as if it began at offset 0, lazily, touching only the .rete bytes a query needs and never the prefix. Pair with detect_polyglot_base to find base.

SearchView

A .rete opened for search only: the dictionary’s chunk directories, and the TEXT_INDEX / pyramid deferred behind their lazy slots. Nothing else.

SliceReader

A RangeReader over an in-memory byte slice (tests, embedded files).

SummaryView

A lightweight, overview-only view of a file: the pyramid summary graph plus just enough dictionary to label predicates. Fetched via ranges without touching the (large) triple index — the “load the coarse graph first” path from SPEC.md §7.2.

Constants

DEFAULT_BLOCK

Default block size: 64 KiB — large enough to swallow a dictionary chunk or an index tile in one fetch, small enough to keep over-fetch modest.

DEFAULT_CACHE_CAP

Default cap on resident cached bytes: 256 MiB. Large enough that a working set (the tiles + dictionary chunks a query family touches) stays warm; small enough that a full sweep of a multi-GB file leaves plenty of the 32-bit wasm address space for the decompressed structures built on top of these bytes. At the auto-tuned 128–512 KiB block sizes this is 512–2048 resident blocks, so the eviction scan is trivial.

POLYGLOT_DIGITS

Fixed decimal width of the offset that follows POLYGLOT_MARKER.

POLYGLOT_MARKER

The marker a polyglot (HTML + .rete) file carries in its first bytes so a reader can locate the embedded .rete without knowing the file size: ASCII RETE-BASE: followed by POLYGLOT_DIGITS zero-padded decimal digits — the byte offset where the .rete begins. It is emitted inside an HTML comment so browsers ignore it, and it sits within the first header window a reader fetches.

Traits

RangeReader

Something that can serve arbitrary byte ranges of a .rete resource.

Functions

auto_block

Pick a BlockCacheReader block size from the file length: bigger files get bigger blocks so a remote query makes far fewer (but larger) round trips — 128 KiB ≤ 10 MB, 256 KiB ≤ 100 MB, 512 KiB above. The over-fetch is modest next to the round-trip latency it saves on a high-latency link (S3/CDN). Shared by the CLI and the wasm client so both size identically; the file length is known for free from the opening HEAD / Content-Range.

detect_polyglot_base

If head (the first bytes of a resource whose byte 0 is NOT the RETE magic) carries a POLYGLOT_MARKER, return the byte offset of the embedded .rete.

read_card_and_build_info_ranged

Fetch the metadata (Dataset Card) and build-info sections via a RangeReader in the fewest requests: the 1 KiB header, then — because the writer lays the build-info section immediately after the metadata — one coalesced range covering both. A file with only one of the two costs the same two requests; a file with neither costs just the header read. This keeps “card + build conditions” within the CARD tier’s 1 header + 1 range budget instead of adding a third request.

read_card_and_build_info_with_header

read_card_and_build_info_ranged, also returning the parsed Header it had to read anyway.

read_metadata_ranged

Fetch only the metadata section (the opaque Dataset Card blob) via a RangeReader: read the 1 KiB header, then the metadata byte range — nothing else. This is the index-free CARD tier of the exploration model: a remote/S3 client learns the dataset’s self-description in two small range requests, never touching the dictionary, index, or pyramid. Returns None when the file carries no metadata.

read_schema_coherence_ranged

Dictionary-free Tier-0 coherence read. Fetch only the header and the pyramid-meta range (2 small range reads) and run schema_coherence over the schema pyramid — never touching the dictionary (which a literal-heavy file makes large) or the triple index. Ok(None) if the file ships no pyramid.

read_schema_summary_ranged

The schema summary (per-class histogram + class relations at the finest level) read over a RangeReader from the schema pyramid alone — the index-free, range-readable source for a Schema view of a remote graph. Returns (classes, relations) with classes = [(class_iri, count)] and relations = [(s_class, predicate, o_class, count)]; None when the file has no schema pyramid. Like read_schema_coherence_ranged, it reads only the trailing schema block, so it stays flat at any graph size.

read_text_index_token_table_len_ranged

Byte length of the TEXT_INDEX section’s leading token table, from a header a caller has already parsed — the prefix range a first Rete::text_search fetches, and the honest figure to quote as its cost (header.text_index_len counts the postings blob too and overstates it several-fold). None when the file carries no text index, or its first bytes could not be read.