Struct RecursiveCharacterSplitter 

pub struct RecursiveCharacterSplitter { /* private fields */ }

Recursive character splitter

Splits text by separator priority, recursively trying smaller separators.

Implementations

impl RecursiveCharacterSplitter

pub fn new( chunk_size: usize, chunk_overlap: usize, ) -> RecursiveCharacterSplitter

Create a new recursive character splitter

chunk_size is clamped to a minimum of 1: a value of 0 — whether passed directly or via a user-supplied separator list that shrank the effective window to nothing — used to make a single character always exceed the quota and recurse forever, blowing the stack.

pub fn with_defaults() -> RecursiveCharacterSplitter

Create with default parameters (chunk_size=1000, chunk_overlap=200)

pub fn with_separators( self, separators: Vec<String>, ) -> RecursiveCharacterSplitter

Set custom separators

Trait Implementations

impl TextSplitter for RecursiveCharacterSplitter

fn split_text(&self, text: &str) -> Vec<String>

Split text into chunks

fn split_document(&self, document: &Document) -> Vec<Document>

Split a document into smaller documents