Expand description
§Chunkers
This module is aimed and the process of “chunking” or also referred to as “text splitting”. Wondering what that is ? put shortly its a set of methods in order to break down text into smaller chunks / pieces. This is done to either keep embedded text short and consise or also to ensure that what your trying to embed isnt larger than the token limit the embedding mode has set out.
Structs§
- Character
Chunker - Token
Chunker - Chunkers
Enums§
- Token
Chunking Error - Chunkers