pub fn smart_chunk(
text: &str,
cfg: &ChunkConfig,
backend: &Arc<Backend>,
) -> Result<Vec<Chunk>, ChipError>Expand description
Chunk text; offsets in the result refer to the normalized text. See
smart_chunk_mapped for original-text spans and ChunkConfig for semantics.