Struct SemanticChunker 

pub struct SemanticChunker<E: Embedder> {
    pub similarity_threshold: f32,
    pub max_chunk_size: usize,
    /* private fields */
}

Semantic chunker that groups sentences by embedding similarity

Fields

similarity_threshold: f32

Similarity threshold (0.0 to 1.0) - chunks split when similarity drops below this

max_chunk_size: usize

Maximum chunk size in characters

Implementations

impl<E: Embedder> SemanticChunker<E>

pub fn new( embedder: E, similarity_threshold: f32, max_chunk_size: usize, ) -> Self

Create a new semantic chunker

Trait Implementations

impl<E: Embedder> Chunker for SemanticChunker<E>

fn chunk(&self, document: &Document) -> Result<Vec<Chunk>>

Split document into chunks

fn estimate_chunks(&self, document: &Document) -> usize

Estimate chunk count without materializing