Module index 

Indexing pipeline

File scanning, parsing, and chunking for document indexing.

Re-exports

pub use ast_chunker::chunk_semantic;

pub use ast_chunker::ChunkType;

pub use ast_chunker::SemanticChunk;

pub use ast_chunker::SemanticChunker;

Modules

ast_chunker

AST-aware semantic chunking

Structs

Chunk

Document chunk

EmbedProgress

Embedding progress

EmbedStats

Embedding statistics

ScanOptions

Scan options

ScanResult

Scan result

Constants

CHUNK_OVERLAP_CHARS

CHUNK_OVERLAP_TOKENS

CHUNK_SIZE_CHARS

CHUNK_SIZE_TOKENS

Chunking configuration

Functions

chunk_by_chars

Character-based chunking (fallback)

embed_documents

Generate embeddings for documents with smart caching

extract_title

Extract title from markdown content

handelize

Normalize path for storage (handelize)

scan_files

Scan directory for files matching pattern