Expand description
Traits in Swiftide allow for easy extendability
All steps defined in the indexing pipeline and the generic transformers can also take a trait. To bring your own transformers, models and loaders, all you need to do is implement the trait and it should work out of the box.
Traitsยง
- Batchable
Transformer - Transforms batched single nodes into streams of nodes
- Chunker
Transformer - Turns one node into many nodes
- DynClone
- This trait is implemented by any type that implements [
std::clone::Clone]. - Embedding
Model - Embeds a list of strings and returns its embeddings. Assumes the strings will be moved.
- Loader
- Starting point of a stream
- Node
Cache - Caches nodes, typically by their path and hash Recommended to namespace on the storage
- Persist
- Persists nodes
- Simple
Prompt - Given a string prompt, queries an LLM
- Sparse
Embedding Model - Embeds a list of strings and returns its embeddings. Assumes the strings will be moved.
- Transformer
- Transforms single nodes into single nodes
- With
Batch Indexing Defaults - Allows for passing defaults from the pipeline to the batch transformer Required for batch transformers as at least a marker, implementation is not required
- With
Indexing Defaults - Allows for passing defaults from the pipeline to the transformer Required for batch transformers as at least a marker, implementation is not required