pub fn chunk_text(
text: &str,
chunk_chars: usize,
overlap_chars: usize,
terminators: &[char],
) -> Vec<String>Expand description
Split text into chunks of at most chunk_chars characters. Paragraphs
(separated by blank lines) are kept whole when they fit; longer ones are
split on sentence boundaries, and sentences longer than a chunk on word
boundaries. Each chunk after the first starts with up to overlap_chars
characters from the end of the previous one, cut at a word boundary.