Expand description
SQLite side of the store: file bookkeeping and chunk payloads.
The vector index only ever holds (chunk id, quantized vector). Everything
a caller actually wants back — the text, the path, the line span — lives
here and is looked up by chunk id after the search returns.
Modules§
- class
- The classes a not-indexed row falls into (2.3), and the dummy list.
Structs§
- Acceptance
- A person’s decision about one file.
- Chunk
Row - A chunk row joined with its file’s path — what a search hit resolves to.
- Corpus
- Everything the
Inside the indexpage shows about one store. - EdgeEnd
- One end of a traversal: the symbol reached, and the edge that reached it.
- Facets
- What a listing spans: the distinct file extensions in it, and the total number of lines across every file.
- File
Kind - One kind of file on the prose panel: how many there are and, for a kind with a unit, how many of that unit they hold.
- File
Lines - FileRow
- One row per indexed file: path, bytes, chunks, the last line any chunk of it covers, and when it was last indexed.
- Filled
- What a reader found for one row: the usage, or only why there is none.
- Image
Row - An indexed image: its file’s path and the pixel size.
- Incoming
- One edge into a name, with the definitions of that name it resolves to.
- Kind
Count - Language
Coverage - What the graph covers, per language, read from the rows themselves.
- Language
Lines - Ledger
Fact - One ledger row, every column a figure is made of.
- Misses
- The two figures for what semlith did not answer.
- Month
Count - Named
Edge - One settled dependency edge, by name, for the community pass.
- NewRetrieval
- One retrieval, before it has a timestamp or a place in the chain.
- Past
Symbol - A definition a re-index replaced.
- Read
File - A file the store read at or after
since, and what it held before. - Refused
- One file or folder that was not indexed, and why.
- Report
Row - One ledger row as the reports count it (0.42.0).
- Retrieval
- One recorded retrieval.
- Retrieval
Row - One recorded retrieval, as the access report attaches them.
- RowUsage
- A row’s usage columns as they are stored. Every figure is optional: a client that logs the model and not the tokens gets the model.
- Savings
- What the ledger adds up to.
- Session
Row - One session’s whole ledger, as the per-session table reads it.
- Stamp
- What a file looked like when it was read for the store, and the one rule every surface judges “edited since it was indexed” by.
- Symbol
Row - A symbol row joined with its file’s path — what a graph answer resolves to.
- SyncRow
- One ledger row as ledger sync sends it: what was retrieved, never the query text, which this struct does not carry.
- Unfilled
- A ledger row whose usage has not been looked for, as the readers in
crate::usageneed it to find the model request that made the call. - Unresolved
- An edge whose target this store has no definition for.
- Writing
- Permission to write, given back when it goes out of scope.
Enums§
- File
Sort - The column a Files listing is ordered by.
- Ledger
Scope - Which rows of the ledger a count is about.
Constants§
- AGENT_
RETRIEVAL - The rows that are an agent’s retrievals, as SQL: what
ledger::figurescounts by default. Not a refund (raw-read), not the chain’s own bookkeeping (a note, a refused write, an accept or revoke), and not a person at the portal or the CLI. Every per-store reader below takes it, so a store’s card and its report say what the Ledger page says. - BASIS_
BYTES - What the whole-file side of a row was counted from: the file’s bytes.
- BASIS_
TEXT - What the whole-file side of a row was counted from: the text a reader extracted from it, counted the way the excerpt was.
- CODE_
CONTEXT - The first format whose code chunks embed the definition they sit inside.
- DEFINITION_
CHUNKS - The first format whose Markdown chunks are cut at their headings.
- FIRST_
IMPORT_ KEY - Meta key present while a store’s first import is under way (#210).
- FORMAT_
KEY - FORMAT_
VERSION - The store layout this binary understands.
- NOTE_
TOOL - The
toolof a ledger note: a row the chain carries that is not a retrieval, and no total counts. - REFUSED_
TOOL - The
toolof a call refused because the agent key is read-only;querynames the tool. Not a retrieval, so no total counts it either. - SHARDED_
FORMAT - The first format that keeps its vectors in shards.
- TURBOVEC_
V8 - The first format whose vector files may be turbovec’s v8.
- USAGE_
COLUMNS - The usage columns, from column
atof a row onwards, in the orderUSAGE_COLUMNSlists them.Noneuntil a reader has looked.
Functions§
- accept
- acceptance
- acceptances
- Every acceptance, for counts and for the folders the walk must not prune.
- all_
hashes - Every stored path’s recorded hash, read once for a whole index pass so its prepare stage can tell an unchanged file without the database.
- all_
paths - bounded
- Run
workoverconns, interrupting their queries oncelimitpasses. - builtin_
targets - How many distinct call and import targets with no definition here are the
language’s own builtins or standard library, and so are not in
unresolved_targets. - busiest_
names - The names with the most edges in or out, busiest first, each counted once on its own index. For the graph page’s choice of where to open: a name with seven hundred definitions is one candidate, not seven hundred.
- chunk
- chunk_
count - Every chunk row, durable or not: what the sidecar can hold a record for.
- chunk_
ids_ of - Drop a file and its chunks, returning the chunk ids so the caller can evict them from the vector index too. The ids of a file’s chunks.
- chunking
- Which chunking rule this store’s chunks were cut by, in the words
statsand the portal print. - chunks_
by_ month - Chunks by the month their file was indexed, oldest first.
- chunks_
overlapping - Every chunk of one file whose lines overlap
start..=end, in file order. - community_
edges - Every
callsandimportsedge whose target this store defines, up tolimit, with the total so a reader is told when the list was cut. - corpus
- Measure one store, for the
Inside the indexpage. - coverage_
by_ language - Per-language coverage for every language the store holds.
- delete_
file - delete_
file_ unretired delete_filewithout keeping the file’s definitions as history: for a stopped run’s undo, whose files were never the store’s to remember.- delete_
graph - Remove one file’s symbols, and with them the edges that leave them.
- drop_
insert_ trigger - Drop the insert trigger an older semlith’s schema puts back when it opens this store, inside the writer’s transaction so it cannot come back before the commit. With it in place every chunk would be indexed twice.
- durable_
chunks - Chunks whose file has a committed hash — the ones whose vectors are on disk.
- edge_
counts - How many edges touch
name, and how many of them are certain, counted on the indexes alone:(resolved, total). - edges_
in - What points at
name: callers, importers, references in. - edges_
in_ first - At most
limitofedges_in, the extractor’s certain edges first. - edges_
in_ resolved - What points at
name, each edge resolved against the name’s definitions the wayedges_outresolves it from the other end. - edges_
out - What the symbols named
namepoint at: callees, imports, references out. - edges_
out_ short_ of_ hubs edges_outwithout the edges whose target has more thanhubdefinitions, dropped before their candidates are joined.- edges_
out_ to edges_outrestricted to edges whose target is one oftargets, settled exactly asedges_outsettles them.- end_
first_ import - The first import ran to its end: later passes write as any other does.
- excerpt_
tokens_ by_ tool (tool, excerpt_tokens)for every ledger row that names its tool, the newestlimitof them: what the Agents page’s typical answer sizes are the median of.- file_
count - How many indexed files match
groups. - file_
facets - file_
hash - Content hash recorded for
path, if we have indexed it before. - file_
rows - The first
takematching files in the requested order. - file_
stamps (bytes, indexed_at)for each ofpathsthe store knows.- files_
ending_ with - Indexed files whose path ends with
suffix, most specific first. - files_
indexed_ since - Files this store read since
since, newest first, with when it read them. - files_
read_ since - Every file read at or after
since, newest first, with no cap — the change brief’s “Files re-read” was 200 per store summed (R3). - fill_
usage - Write a row’s usage. Every row sharing its
query_idis the same call, written once per store that answered, so all of them are filled. - filtered_
chunk_ ids - Ids of every chunk whose file matches
groups. - filtered_
image_ ids - The image ids whose file matches
groups, for a filtered search. - filtered_
paths - Indexed paths whose file matches
groups, in path order. - first_
import - Whether this pass is part of a store’s first import, starting one when the store is empty.
- format
- Which layout this store’s vectors are in.
- free_
bytes - Pages SQLite holds free inside the file, in bytes: what a vacuum gives back before anything else is dropped.
- get_
meta - graph_
input - A file’s id and its chunks’ line spans, for writing its graph again.
- graph_
stats (symbols, edges)— the graph’s size, forstatsand the portal.- grep_
cost - The total bytes of every file a grep for
namewould have to read. - heal_
stamps - Bring an unchanged file’s
bytesandindexed_atup to what is on disk, when the file’s size or mtime has moved past the row. - held_
text - The text the store holds for
path, stitched by line from its chunks. - history_
bytes_ before - Roughly what the definitions retired before
beforetake: their text plus a row’s fixed cost. An estimate forstats, not an accounting — SQLite packs rows into pages and the count cannot see that. - history_
count_ before - How many definitions were retired before
before. - hold_
automerge - Stop FTS5 merging segments while a first import writes (#210).
- holds_
path - Whether this store has
pathindexed. - image
- One image row, joined with its file’s path.
- image_
count - How many images this store holds.
- image_
ids - Every image id, for the ids a filtered search may consider.
- image_
ids_ of - The image ids belonging to
path, before its row is deleted. - image_
rows - Every indexed image, for the decisions table’s “read as image” rows.
- insert_
chunk - insert_
edge - insert_
file - insert_
image - insert_
symbol - is_
interrupted - Whether an error is a query
boundedinterrupted. - is_
pinned - Whether this connection is holding a run’s snapshot.
- keyword_
search - keyword_
search_ within - The keyword list inside a set of chunk ids, for a filtered search that has the set already.
- last_
write - The newest
indexed_atthis store holds, orNonewhen it holds nothing. - ledger_
break - Re-walk the chain and return the id of the first row that does not verify.
- ledger_
clients - How many retrievals each client made, most first.
- ledger_
facts - Every ledger row with
since <= at < until, oldest first. - ledger_
head - The hash of the newest ledger row: what
--verifychecks the chain to. - ledger_
keys - The distinct retrievals this store holds, as keys that can be unioned across stores.
- ledger_
latency - Warm query latency as the ledger recorded it: p50 and p95, in microseconds.
- ledger_
misses - What the ledger says about the questions semlith did not answer.
- ledger_
note - Re-anchor the chain after a row that does not verify, by appending a note row onto the current head (0.35.0’s Re-verify with repair).
- ledger_
repair - Append a note for every unacknowledged break, first to last, and say which rows they were. Bounded, so a chain broken at every row is not a loop.
- ledger_
retrievals - Every retrieval since
since, newest first. - ledger_
rows - How many rows the chain holds, notes included: what a verify walked.
- ledger_
savings - The one savings figure, with the denominators that make it readable.
- ledger_
savings_ since - The savings, over retrievals recorded at or after
since. - ledger_
sessions - Every session this store recorded, newest last-seen first.
- ledger_
totals (queries, clients, excerpt tokens, whole-file tokens)over the whole///(queries, clients, excerpt tokens, whole-file tokens)over the whole ledger — what the Ledger page shows.- ledger_
zero_ hit_ queries - Queries that found nothing, commonest first.
- ledger_
zero_ hit_ queries_ since - The questions that found nothing, at or after
since. - mark_
synced - Record that these rows reached the org’s ledger.
- mark_
vector_ format - Move a format-4 store to
TURBOVEC_V8before this binary writes to it. - matching_
files - How many indexed files
groupsselects. - mtime_
ns - A file’s modification time in nanoseconds since the epoch.
- names_
with_ several_ definitions - How many names this store defines more than once.
- newest_
chunk - The highest chunk id, or 0 for none. One step down the rowid b-tree.
- open
- Open a store’s database, laying down or upgrading its schema first.
- owner_
of - The type or module a definition’s
qualifiedname puts it in:FleetforFleet::search_preferring,storeforstore::edges_out. - path_
bounds - The first and last indexed path in sort order. Their common directory is
the common directory of every path, which is the root a report prints
paths against when the registry names none (a store opened by
--store). - pending_
among - Of
ids, the chunks whose file is still being embedded, with their text: what a search mid-run embeds on the query path. - pending_
share - The share of the store’s chunks still being embedded, or
Nonewhen nothing is: one indexed probe on every search, and the counts only when the probe finds a run in progress. - pin_
snapshot - Hold this reader at the store as it is now, until
unpin_snapshot. - prune_
history - Drop the definitions retired before
before(unix seconds). The retention is the caller’s; seecompact. - read_
only - Make this connection refuse writes until a writer says otherwise.
- readable
(files, chunks, indexed bytes)Whether this store can still be read, as cheaply as that can be asked.- reclaim
- Give the database’s dead pages back to the filesystem.
- record_
counted record_retrieval, saying what the whole-file side was counted from.- record_
legacy_ retrieval - Append a row in the shape 0.14.0 wrote, for the test that proves a store holding both kinds still verifies.
- record_
retrieval - Append one retrieval, chained to the row before it.
- record_
v1_ retrieval - Append a row the way 0.15.0 to 0.41.0 wrote it — chain v1, no basis, no head — for the test that proves a store holding v1 and v2 rows verifies.
- refusal
- One not-indexed row, by exact path.
- refusals
- Every not-indexed row, reviewable first, then by class and path.
- refuse
- Record that
pathwas not indexed. The first time it was seen is kept. - release_
automerge - Put back the
automergesettinghold_automergerecorded, if any. - report_
rows_ since - Every ledger row recorded at or after
since, oldest first. - retrieval_
at - When a ledger row was written, by id.
- retrievals
- The most recent
limitretrievals, newest first. - revoke
- salt
- The per-store salt fingerprints are keyed with, made on first need.
- salt_
if_ any - The salt, when a store has one. A read-only caller cannot make one, and a store without one has no acceptances to compare against.
- secure_
delete_ fts - Turn on FTS5’s secure delete for
chunks_fts, if it is not on: a removed row’s terms then leave the index’s segments at once rather than at their next merge, so a file refused for a secret leaves no searchable term behind. A setting the index keeps; older binaries read it unchanged. - set_
file_ graph - Record what the parser made of a file:
parsed,timeout, ornonefor a language semlith carries no grammar for. - set_
file_ units - Record a file’s units (see
chunk::COUNTED), by row id or, for a file whose row is already there, by path. - set_
meta - set_
read_ mtime - Record the modification time a file’s read saw. See
Stamp::fresh. - set_
units_ by_ path - stats
- symbols_
by_ names - Symbols defined in files matching
groupswhose name is one ofnames. - symbols_
in_ files - Every symbol defined in each of
paths. - symbols_
named - Every definition of
name, across the whole store. - symbols_
named_ counted symbols_namedwith code first — definitions before the headings, keys and selectors that share the name — and how many there are in all.- symbols_
past_ count - How many definitions this store has retired, over all names.
- symbols_
past_ named - What a symbol used to be, newest first.
- symbols_
scoped - Symbols to draw for a scope, newest-defined first within each file.
- text_of
- Everything this store holds for one file, its chunks in order, joined.
- uncounted
- Stored files of a counted format whose units were never recorded — written by a binary older than 0.34.0. The index pass reads each one again for its count alone.
- unfilled
- Rows written by an AI client since
since(unix seconds) with no usage yet, oldest first.cliandportalrows have no model behind them, and neither has araw-readrow, which the steering hook writes for a read the agent made without semlith. - unpin_
snapshot - End
pin_snapshot’s read transaction: the reader sees the run’s commit. - unrefuse
- Forget that
pathwas not indexed: it has been, or it is gone. - unresolved_
out - Targets of
name’s edges that the store holds no definition for. - unresolved_
targets - Call targets no definition in this store satisfies, commonest first, and how many distinct ones there are.
- unsynced
- Rows written at or after
sinceand not yet sent, oldest first, at mostlimit. - unsynced_
count - How many rows since
sinceare still to send. - usage_
at - usage_
rows - Every row with usage, keyed as
ledger_keyskeys it, for totals per client and per model that count a cross-store call once. - where_
defined - Where each of these names is defined, for the hub lines under a community.
- write_
extraction - Write one file’s extraction: its symbols, then its edges by source name.
Type Aliases§
- Chunk
Span - A chunk’s first line, last line and id.
- Symbol
Span - One definition’s span and identity:
(start, end, name, kind).