Struct CodeChunker 

pub struct CodeChunker { /* private fields */ }

Language-aware code chunker.

Implementations

impl CodeChunker

pub fn new(language: Language) -> Self

Create a chunker for language with default parameters.

pub fn with_fallback_window(self, window: usize) -> Self

Configure the fallback RecursiveCharSplitter window size.

pub fn with_min_chunk_chars(self, min: usize) -> Self

Discard chunks shorter than min characters.

pub fn language(&self) -> Language

The language this chunker was constructed with.

pub fn chunk(&self, text: &str, doc_id: usize) -> Result<Vec<Chunk>, RagError>

Split text into chunks appropriate for its language.

Trait Implementations

impl Default for CodeChunker

fn default() -> Self

Returns the “default value” for a type.