Skip to main content

chunk_text

Function chunk_text 

Source
pub fn chunk_text(
    text: &str,
    chunk_size: usize,
    overlap: usize,
    strategy: ChunkStrategy,
) -> Result<Vec<TextChunk>, ChunkError>
Expand description

Split text into chunks using the specified strategy.

  • chunk_size: maximum number of characters per chunk.
  • overlap: number of characters shared between consecutive chunks.
  • strategy: splitting strategy.

Returns a deterministic sequence of chunks. Same input + params → same output.