Expand description
Traits in Swiftide allow for easy extendability
All steps defined in the indexing pipeline and the generic transformers can also take a trait. To bring your own transformers, models and loaders, all you need to do is implement the trait and it should work out of the box.
Modules§
- __
mock_ Mock Batchable Transformer - __
mock_ Mock Batchable Transformer_ Batchable Transformer - __
mock_ Mock Batchable Transformer_ Clone - __
mock_ Mock Chunker Transformer - __
mock_ Mock Chunker Transformer_ Chunker Transformer - __
mock_ Mock Chunker Transformer_ Clone - __
mock_ Mock Embedding Model - __
mock_ Mock Embedding Model_ Clone - __
mock_ Mock Embedding Model_ Embedding Model - __
mock_ Mock Loader - __
mock_ Mock Loader_ Clone - __
mock_ Mock Loader_ Loader - __
mock_ Mock Node Cache - __
mock_ Mock Node Cache_ Clone - __
mock_ Mock Node Cache_ Node Cache - __
mock_ Mock Persist - __
mock_ Mock Persist_ Clone - __
mock_ Mock Persist_ Persist - __
mock_ Mock Simple Prompt - __
mock_ Mock Simple Prompt_ Clone - __
mock_ Mock Simple Prompt_ Simple Prompt - __
mock_ Mock Sparse Embedding Model - __
mock_ Mock Sparse Embedding Model_ Clone - __
mock_ Mock Sparse Embedding Model_ Sparse Embedding Model - __
mock_ Mock Transformer - __
mock_ Mock Transformer_ Clone - __
mock_ Mock Transformer_ Transformer
Structs§
- Mock
Batchable Transformer - Mock
Chunker Transformer - Mock
Embedding Model - Mock
Loader - Mock
Node Cache - Mock
Persist - Mock
Simple Prompt - Mock
Sparse Embedding Model - Mock
Transformer
Traits§
- Batchable
Transformer - Transforms batched single nodes into streams of nodes
- Chunker
Transformer - Turns one node into many nodes
- DynClone
- This trait is implemented by any type that implements [
std::clone::Clone]. - Embedding
Model - Embeds a list of strings and returns its embeddings. Assumes the strings will be moved.
- Loader
- Starting point of a stream
- Node
Cache - Caches nodes, typically by their path and hash Recommended to namespace on the storage
- Persist
- Persists nodes
- Simple
Prompt - Given a string prompt, queries an LLM
- Sparse
Embedding Model - Embeds a list of strings and returns its embeddings. Assumes the strings will be moved.
- Transformer
- Transforms single nodes into single nodes
- With
Batch Indexing Defaults - Allows for passing defaults from the pipeline to the batch transformer Required for batch transformers as at least a marker, implementation is not required
- With
Indexing Defaults - Allows for passing defaults from the pipeline to the transformer Required for batch transformers as at least a marker, implementation is not required