Struct TextChunk 

pub struct TextChunk {
    pub id: usize,
    pub text: String,
    pub char_offset: usize,
    pub char_length: usize,
    pub document_id: Option<String>,
    pub has_overlap: bool,
    pub overlap_info: Option<(usize, usize)>,
}

A chunk of text with metadata

Fields

id: usize

The chunk ID

text: String

Text content of the chunk

char_offset: usize

Character offset from the beginning of the original document

char_length: usize

Length of the chunk in characters

document_id: Option<String>

Original document this chunk belongs to

has_overlap: bool

Whether this chunk overlaps with adjacent chunks

overlap_info: Option<(usize, usize)>

Overlap information (start and end overlap lengths)

Implementations

impl TextChunk

pub fn new( id: usize, text: String, char_offset: usize, document_id: Option<String>, ) -> Self

Create a new text chunk

pub fn with_overlap( id: usize, text: String, char_offset: usize, document_id: Option<String>, overlap_start: usize, overlap_end: usize, ) -> Self

Create a chunk with overlap information

pub fn char_interval(&self) -> CharInterval

Get the character interval for this chunk in the original document

pub fn core_text(&self) -> &str

Get the core text without overlaps

Trait Implementations

impl Clone for TextChunk

fn clone(&self) -> TextChunk

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TextChunk

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.