Modules§
- loaders
- The
loaders
module provides functionality for loading files from a specified directory. It includes theFileLoader
struct which is used to filter and stream files based on their extensions. - persist
- Storage implementations for persisting data
- transformers
- Various transformers for chunking, embedding and transforming data
Structs§
- Pipeline
- A pipeline for indexing files, adding metadata, chunking, transforming, embedding, and then storing them.