Struct HierarchicalChunker 

pub struct HierarchicalChunker { /* private fields */ }

Hierarchical text chunking following LangChain RecursiveCharacterTextSplitter approach with semantic boundary preservation for optimal RAG performance

Implementations

impl HierarchicalChunker

pub fn new() -> Self

Create a new hierarchical chunker with default separators

pub fn with_separators(separators: Vec<String>) -> Self

Create chunker with custom separators

pub fn with_min_size(self, min_size: usize) -> Self

Set minimum chunk size

pub fn chunk_text( &self, text: &str, chunk_size: usize, overlap: usize, ) -> Vec<String>

Split text into semantically coherent chunks

pub fn find_sentence_boundary( &self, text: &str, start: usize, preferred_end: usize, ) -> Option<usize>

Advanced sentence boundary detection

Trait Implementations

impl Default for HierarchicalChunker

fn default() -> Self

Returns the “default value” for a type.