Module chunking 

Available on crate feature rag only.

Document chunking strategies.

This module provides the Chunker trait and three implementations:

• FixedSizeChunker — splits by character count with configurable overlap
• RecursiveChunker — splits hierarchically by paragraphs, sentences, then words
• MarkdownChunker — splits by markdown headers, preserving header context

Structs

FixedSizeChunker

Splits text into fixed-size chunks by character count with configurable overlap.

MarkdownChunker

Splits text by markdown headers, keeping each section as a chunk.

RecursiveChunker

Splits text hierarchically: paragraphs → sentences → words.

Traits

Chunker

A strategy for splitting documents into chunks.