Skip to main content

Module chunk

Module chunk 

Source
Expand description

Canonical chunk types and chunk_id derivation.

chunk_id is a deterministic SHA-256 over the canonical text plus the original location and the chunker version. The exact preimage is fixed by spec - anything else would let two runs of the same chunker produce different IDs for the same content, which breaks reproducibility.

Structs§

ChunkInput
One chunk’s worth of input that the writer needs to produce a v1 file.

Functions§

chunk_id
Compute the canonical chunk_id for the given inputs.
validate_chunk
Validate a ChunkInput against the embedding dimension declared in the manifest. Returns the validated reference for ergonomics.