Expand description
Text splitting utilities.
This module provides text splitting functionality including:
- Splitting by regex separators
- Recursive syntax-aware chunking
Structs§
- Chunk
- A chunk of text with its range and position information.
- Custom
Language Config - Configuration for a custom language with regex-based separators.
- Output
Position - Output position information with character offset and line/column.
- Recursive
Chunk Config - Configuration for a single chunking operation.
- Recursive
Chunker - A recursive text chunker with syntax awareness.
- Recursive
Split Config - Configuration for recursive text splitting.
- Separator
Split Config - Configuration for separator-based text splitting.
- Separator
Splitter - A text splitter that splits by regex separators.
- Text
Range - A text range specified by byte offsets.
Enums§
- Keep
Separator - How to handle separators when splitting.