Skip to main content

CHUNK_OVERLAP_TOKENS

Constant CHUNK_OVERLAP_TOKENS 

Source
pub const CHUNK_OVERLAP_TOKENS: usize = 50;
Expand description

Token overlap between consecutive chunks.