Module format 

Stable file-format, reader, and in-memory build API.

Structs

BuildInfoPlan

Everything needed to splice a build-info section into a finished file without holding the file in memory: the rewritten header, where the section goes, and where the bytes that do not move resume.

BuildStats

Counts describing an assembled file, for status lines and UIs.

ByteRange

A byte range in the .rete file image.

CardAppendPlan

What placing a new card after a file’s data does to its layout — see “The card after the data” in the module docs.

CardRewritePlan

What a card rewrite does to a file’s layout.

DeferredMetadata

A metadata payload whose final form needs the written (deduplicated) counts. See IntoMetadata.

DumpPlan

What a filtered dump will cost, before it runs — see Rete::dump_plan.

FinalCounts

The deduplicated counts of the file being written — what the header records, and therefore what any embedded metadata has to report.

Header

Decoded file header. All multi-byte fields are little-endian on disk. The *_offset / *_len fields are a convenience view over the section directory (populated from it on parse, emitted back to it on serialize).

LayoutSegment

One labelled byte region of a .rete file image (see Rete::file_layout). kind is a stable machine tag: header, metadata, dictionary, directory, tile, pyramid, named-graphs.

Rete

A read-only, in-memory view over a .rete file image.

ScanPlan

What a routed scan of one pattern will touch, from the tile directories alone — see GraphIndex::scan_plan.

Section

One parsed/encoded section-directory entry: a typed (offset, length) into the file, plus 16 bits of per-section flags (reserved).

TripleProvenance

Why a triple-pattern result is present in the file.

Enums

FileError

GraphLayout

How a build stores the named graphs (the default graph is unaffected).

HeaderError

IngestError

Why an ingest failed. Parser errors from oxttl/oxrdfxml and I/O errors are flattened to their Display string rather than wrapped: the variants then carry no foreign types, which keeps the error usable from the wasm bindings and stable across upstream parser upgrades.

PyramidLayout

What a pyramid-meta section carries besides the community summary.

QuotedTripleSurface

Which surface a << … >> in Turtle/TriG input is read as — the input half of rete export --quoted-triple-syntax, same vocabulary, same two values.

SectionKind

A top-level file section, addressed by SectionKind in the header directory.

Constants

CODEC_NONE

No compression.

CODEC_ZSTD

zstd compression (per section).

CURRENT_FORMAT_VERSION

Current format generation written by this crate.

HEADER_LEN

Fixed header size in bytes.

MAGIC

Magic bytes at offset 0: ASCII RETE.

MIN_STABLE_READ_VERSION

Oldest stable format generation accepted by this reader.

RDF_TYPE

rdf:type — the predicate that assigns a class to a resource.

REWRITE_CHUNK

How the rewriter moves bytes: large enough to keep a disk streaming, small enough that the peak memory of a 26 GB rewrite is this and nothing else.

Traits

IntoMetadata

What a build’s metadata callback may return.

Functions

append_sections

Append LABEL_INDEX / SCHEMA sections (kinds 8, 9) to a finished file image, just before its 4-byte footer, and return the new image.

assemble_dataset

Assemble a complete .rete file image from parsed quads: one shared dictionary, the default-graph index, one index per named graph, and the community pyramid. metadata is the opaque metadata-section payload (the CLI puts a JSON Dataset Card there); pass &[] for none — that is byte-identical to a metadata-free build.

assemble_dataset_with

Like assemble_dataset, but the metadata payload is derived from the BuildStats and the source quads while they are still resident — for metadata that describes the graph (the Dataset Card). Returning an empty Vec is byte-identical to a metadata-free build; return a DeferredMetadata for a payload that must also carry the file’s deduplicated counts, which exist only once the indexes are built.

assemble_dataset_with_layout

Like assemble_dataset_with_perms, storing the named graphs in graphs layout. GraphLayout::PerGraph reproduces it exactly.

assemble_dataset_with_opts

Like assemble_dataset_with, but with_pyramid = false skips the Louvain community pyramid entirely — no pyramid section is written (header length 0). SPARQL / SHACL / triple / reachability queries don’t use the pyramid, so a pyramid-less file is fully queryable and markedly smaller (the pyramid is the largest section on highly-connected graphs). Only the community / summary / progressive paths need it.

assemble_dataset_with_opts_algo

Like assemble_dataset_with_opts, but selects the community PyramidAlgo (the in-memory build path for rete build --pyramid-algo …).

assemble_dataset_with_perms

Like assemble_dataset_with_opts_algo, but writes only the permutations in perms (rete build --permutations 3). Every other byte of the file is unchanged; crate::index::PermSet::ALL reproduces the default build exactly, down to the header byte.

attach_build_info

Attach (or replace) a build-info section in a finished file image, returning the new image. Layout and hash semantics: plan_build_info.

card_append_hash

Fold the content hash the file will have once plan is committed with metadata as its card: the card from memory, every other hashed section read from src (the file as the plan found it) in REWRITE_CHUNK slices. progress gets the bytes read so far. The build record does not take part — it is outside the hash — which is what lets a caller seal it after this, its longest step, and still commit it in the same write.

commit_card_append

Commit plan: write metadata, then build_info (outside the hash, so it may be sealed after card_append_hash), then a footer at the plan’s card offset, cut the file there, sync, and only then rewrite the 1 KiB header with hash and the new section entries, and sync again. Returns the header written.

content_hash_parts

The byte ranges the content hash folds, in the order it folds them: the metadata payload when present, then dictionary, index, pyramid-meta, and — when present — a kind-6 text index and the named graphs. Build info, the self-digested kinds (8 LABEL_INDEX, 9 SCHEMA, 11 predicate-restricted text index) and any section kind this reader does not know are deliberately outside it. The quad index (kind 10) is not a range here either: its digest is the seal at the end of the index container, so it is hashed by reference.

parse

Parse N-Triples text into raw term-token triples, skipping blank/comment lines.

parse_quads

Parse N-Quads text: subject predicate object [graph] . per line.

parse_rdfxml

Parse RDF/XML into canonical N-Triples-token triples via oxrdfxml. This is how most OWL ontologies ship (.rdf/.owl/.xml with an rdf:RDF root) — so rete ingests them directly, no external conversion. (OWL/XML — the non-RDF functional XML serialization — is a different language; convert it with owlready2 first.)

parse_statements

Parse one text input by format name ("nt", "nq", "ttl", "trig", or "rdfxml") into quads (triples land in the default graph).

parse_statements_audited_surface

parse_statements_audited reading << … >> in the given QuotedTripleSurface. This is the entry point rete build and rete validate use for everything but a streamed line-based input, and the one that turns “RDF 1.2 file, RDF-star reader” into an error that names the flag instead of a bare syntax complaint.

parse_turtle

Parse Turtle into canonical N-Triples-token triples via oxttl, reading << s p o >> as an RDF-star quoted triple. parse_turtle_surface chooses.

parse_turtle_surface

parse_turtle in the given QuotedTripleSurface.

plan_build_info

Plan the splice of a build-info section of info_len bytes into a file whose first HEADER_LEN bytes are head and whose total length is file_len.

plan_card_append

Plan placing a card of metadata_len bytes (0 = none) after the data of the file whose first HEADER_LEN bytes are head and whose length on disk is file_len.

plan_card_rewrite

Plan replacing the metadata and build-info sections of a file whose first HEADER_LEN bytes are head and whose length is file_len, with sections of metadata_len and build_info_len bytes (0 = none).

read_build_info

The build-info section of a whole file image, or None if absent. The payload is an opaque application-layer blob (the CLI stores build-conditions JSON there); it is not covered by the content hash — see attach_build_info.

replace_metadata

Replace the metadata section (the Dataset Card blob) of a finished file image, returning the new image. The section is the first one after the header, so every later section’s offset shifts by the length delta.

rewrite_card

Rewrite src into out with metadata as its card and build_info as its build record (either may be empty, which removes the section), copying every other section verbatim and folding the new content hash on the way. Returns the header written at offset 0.

verify

Recompute the content hash from a file image and check it against the header — detects corruption or truncation of the payload sections. The LABEL_INDEX, SCHEMA and predicate-restricted TEXT_INDEX sections (kinds 8, 9, 11) are outside that hash — an older reader’s verify must keep passing on a file that carries them — so each is checked against its own leading digest instead. The QUAD_INDEX section (kind 10) is hashed by reference: its digest is the seal inside the (hashed) index container, checked here.

Type Aliases

RawQuad

A parsed quad: the triple plus an optional graph term (None = default graph).

RawTriple

A parsed triple as three canonical term tokens.

TermTriple

A resolved triple as terms.