Struct Chunk 

pub struct Chunk {
    pub id: Uuid,
    pub doc_id: Uuid,
    pub text: String,
    pub byte_offset: u64,
    pub byte_length: u64,
    pub sequence: u32,
    pub text_hash: [u8; 32],
}

A chunk of text extracted from a document

Documents are split into overlapping chunks for embedding. Each chunk tracks its position within the source document.

Per CP-011: Uses byte-based offsets (not character-based) for accurate slicing back to original document content.

Per CP-001: Chunk ID is STABLE - ID = hash(doc_id + sequence) only. This ensures re-chunking with different parameters produces the same IDs.

Fields

id: Uuid

Unique identifier for this chunk (BLAKE3-16 of doc_id + sequence) - STABLE

doc_id: Uuid

Parent document ID

text: String

The actual text content (canonicalized)

byte_offset: u64

Byte offset within the source document (u64 for large files)

byte_length: u64

Length of this chunk in bytes (u64 for large files)

sequence: u32

Sequence number within the document (0-indexed)

text_hash: [u8; 32]

Hash of the canonicalized text content for verification

Implementations

impl Chunk

pub fn new(doc_id: Uuid, text: String, byte_offset: u64, sequence: u32) -> Self

Create a new chunk with automatic ID generation.

Per CP-001: Chunk ID is STABLE - does NOT include text. This ensures re-chunking with different parameters produces same IDs. Content is verified via text_hash field.

pub fn text_hash_hex(&self) -> String

Get the text hash as a hex string

pub fn approx_tokens(&self) -> usize

Approximate token count (rough estimate: 4 chars per token)

Trait Implementations

impl Clone for Chunk

fn clone(&self) -> Chunk

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Chunk

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for Chunk

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for Chunk

fn eq(&self, other: &Chunk) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for Chunk

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Eq for Chunk

impl StructuralPartialEq for Chunk