Skip to main content

Module chunker

Module chunker 

Source
Expand description

Semantic windowing chunker for RAG indexing.

Splits timeline entries into overlapping windows of ~1.5k tokens, suitable for vector embedding and semantic search via memex.

Vibecrafted with AI Agents by VetCoders (c)2026 VetCoders

Structs§

Chunk
A single chunk ready for vector indexing.
ChunkMetadataSidecar
Structured metadata sidecar persisted alongside each chunk file.
ChunkerConfig
Configuration for the chunker.

Constants§

INTENT_KEYWORDS

Functions§

chunk_entries
Chunk timeline entries into semantic windows with overlap.
chunk_summary
Summary of chunking results.
classify_kind
Classify a set of timeline entries into a canonical Kind.
estimate_tokens
Estimate token count from text length.
format_chunk_text
Format entries into chunk text with metadata header.
is_decision_tag
is_outcome_tag
is_result_line
normalize_key
parse_checklist_task
truncate_signal_line
write_chunks_to_dir
Write chunks as individual .txt files to a directory.