Crate semlith 

semlith — a local semantic cache for AI agents.

Point it at files, get back a store that answers “what part of my corpus is relevant to this question?” in milliseconds, without shipping anything off the machine and without an agent burning tokens reading whole files.

Two pieces of state live side by side in the store directory:

• the vectors — a turbovec TurboQuant index holding only quantized vectors keyed by chunk id. A store created by 0.7.0 keeps them as a directory of shards under index/; every store written before it keeps the single index.tv it already has. See index.
• store.db — SQLite holding the chunk text, its file, and its line span.

A search quantizes the query, gets ids back from the index, then resolves them to text with one SQLite lookup each.

Modules

chunk

Turning a file on disk into embeddable chunks of text.

embed

Which embedding model a store uses, and how to load it.

filter

Narrowing a search to part of the corpus.

fleet

Several stores, one query.

index

The vector side of a store: read from disk only when something needs it, and in pieces small enough to put down again.

lock

One writer at a time per store.

mcp

A stdio MCP server, so an agent can query and maintain the store as tool calls.

store

SQLite side of the store: file bookkeeping and chunk payloads.

watch

Keep a store current while its corpus is being edited.

Structs

Hit

IndexProgress

Where an index run has got to, handed to the callback with each file it starts embedding.

IndexReport

Semlith

Functions

canonical

default_model

Default model: 384-dim, ~52 MB on disk. Measured against the previous default (BGE-small) on a 6260-chunk corpus it scored 16.00 code MRR@10 against 14.84, at a third of the download.

default_store_dir

Default store location: .semlith beside whatever you are indexing.

human_bytes

Byte counts sized to the unit that actually shows a digit — a small corpus reported as “0.0 MB” reads like a bug rather than a small corpus.

model_cache_dir

Where ONNX model weights are cached. Shared across stores — the weights are large and identical for a given model.

store_dirs

The stores a command should use: the --store flags if any were given, else whatever SEMLITH_STORE names, else .semlith in the current directory.