Skip to main content

Module reader

Module reader 

Source
Expand description

Zero-copy view over a .urna byte slice.

The reader does no I/O - callers pass an &[u8] (e.g. backed by an mmap). Parsing validates magic, header checksum, file_size, all section checksums, footer hash, manifest schema, and the presence of every required section.

Section payloads come in three encodings (SECTION_ENCODING_*):

  • raw: the section bytes ARE the canonical payload.
  • zstd: stored compressed; the reader decompresses on demand and returns an owned Cow::Owned buffer.
  • float16 / int8: only valid for the embeddings section; the physical bytes are also the canonical bytes (the runtime dispatches on manifest.dtype).

Section checksums hash the physical bytes as stored. content_hash hashes the decoded bytes so a zstd-compressed corpus and its raw equivalent share the same content_hash (and therefore the same citation URIs).

Structs§

UrnaView

Functions§

validate_slab_values
Reject any NaN/Inf in a fixed-stride vector slab of the given section encoding (raw f32, float16, int8, int4). The canonical embeddings, the embeddings_fp rerank slab and every multimodal band go through this before a kernel ever scores them: a NaN lane would turn a cosine into a NaN score, and a NaN score is what breaks a ranking (found by the mutation fuzz harness through the space bands).