Crate semlith 

semlith — a local semantic cache for AI agents.

Point it at files, get back a store that answers “what part of my corpus is relevant to this question?” in milliseconds, without shipping anything off the machine and without an agent burning tokens reading whole files.

Two pieces of state live side by side in the store directory:

• index.tv — a turbovec TurboQuant index holding only quantized vectors keyed by chunk id.
• store.db — SQLite holding the chunk text, its file, and its line span.

A search quantizes the query, gets ids back from the index, then resolves them to text with one SQLite lookup each.

Modules

chunk

Turning a file on disk into embeddable chunks of text.

embed

Which embedding model a store uses, and how to load it.

lock

One writer at a time per store.

mcp

A stdio MCP server, so an agent can query the store as a tool call.

store

SQLite side of the store: file bookkeeping and chunk payloads.

Structs

Hit

IndexReport

Semlith

Functions

canonical

default_model

Default model: 384-dim, ~52 MB on disk. Measured against the previous default (BGE-small) on a 6260-chunk corpus it scored 16.00 code MRR@10 against 14.84, at a third of the download.

default_store_dir

Default store location: .semlith beside whatever you are indexing.

human_bytes

Byte counts sized to the unit that actually shows a digit — a small corpus reported as “0.0 MB” reads like a bug rather than a small corpus.

model_cache_dir

Where ONNX model weights are cached. Shared across stores — the weights are large and identical for a given model.