Struct CharacterTextSplitter 

pub struct CharacterTextSplitter { /* private fields */ }

Splits text by a single separator string.

After splitting, chunks are merged to stay under chunk_size with chunk_overlap characters of overlap between consecutive chunks.

Implementations

impl CharacterTextSplitter

pub fn new(chunk_size: usize) -> Self

pub fn with_separator(self, separator: impl Into<String>) -> Self

pub fn with_chunk_overlap(self, overlap: usize) -> Self

Trait Implementations

impl TextSplitter for CharacterTextSplitter

fn split_text(&self, text: &str) -> Vec<String>

Split a string into chunks.

fn split_documents(&self, docs: Vec<Document>) -> Vec<Document>

Split documents by splitting each document’s content and producing new documents for each chunk. Metadata is preserved on each chunk.