Module indexer

Module indexer 

Source
Expand description

Document indexing pipeline for the RAG system

This module handles parsing and chunking of various document formats for indexing into the vector database.

Structs§

Indexer
Document indexer for the RAG system

Functions§

find_documents
Find all indexable documents in a directory using WalkDir for performance
index_document
Index a single document and return chunks
is_supported_file
Check if a file is supported for indexing