Module indexing_traits

Traits in Swiftide allow for easy extendability

All steps defined in the indexing pipeline and the generic transformers can also take a trait. To bring your own transformers, models and loaders, all you need to do is implement the trait and it should work out of the box.

Traits

• BatchableTransformer
  Transforms batched single nodes into streams of nodes
• ChunkerTransformer
  Turns one node into many nodes
• DynClone
  This trait is implemented by any type that implements [std::clone::Clone].
• EmbeddingModel
  Embeds a list of strings and returns its embeddings. Assumes the strings will be moved.
• Loader
  Starting point of a stream
• NodeCache
  Caches nodes, typically by their path and hash Recommended to namespace on the storage
• Persist
  Persists nodes
• SimplePrompt
  Given a string prompt, queries an LLM
• SparseEmbeddingModel
  Embeds a list of strings and returns its embeddings. Assumes the strings will be moved.
• Transformer
  Transforms single nodes into single nodes
• WithBatchIndexingDefaults
  Allows for passing defaults from the pipeline to the batch transformer Required for batch transformers as at least a marker, implementation is not required
• WithIndexingDefaults
  Allows for passing defaults from the pipeline to the transformer Required for batch transformers as at least a marker, implementation is not required