Skip to main content

Module store

Module store 

Source
Expand description

SQLite side of the store: file bookkeeping and chunk payloads.

The vector index only ever holds (chunk id, quantized vector). Everything a caller actually wants back — the text, the path, the line span — lives here and is looked up by chunk id after the search returns.

Modules§

class
The classes a not-indexed row falls into (2.3), and the dummy list.

Structs§

Acceptance
A person’s decision about one file.
ChunkRow
A chunk row joined with its file’s path — what a search hit resolves to.
Corpus
Everything the Inside the index page shows about one store.
EdgeEnd
One end of a traversal: the symbol reached, and the edge that reached it.
Facets
What a listing spans: the distinct file extensions in it, and the total number of lines across every file.
FileKind
One kind of file on the prose panel: how many there are and, for a kind with a unit, how many of that unit they hold.
FileLines
FileRow
One row per indexed file: path, bytes, chunks, the last line any chunk of it covers, and when it was last indexed.
Filled
What a reader found for one row: the usage, or only why there is none.
ImageRow
An indexed image: its file’s path and the pixel size.
Incoming
One edge into a name, with the definitions of that name it resolves to.
KindCount
LanguageCoverage
What the graph covers, per language, read from the rows themselves.
LanguageLines
LedgerFact
One ledger row, every column a figure is made of.
Misses
The two figures for what semlith did not answer.
MonthCount
NamedEdge
One settled dependency edge, by name, for the community pass.
NewRetrieval
One retrieval, before it has a timestamp or a place in the chain.
PastSymbol
A definition a re-index replaced.
ReadFile
A file the store read at or after since, and what it held before.
Refused
One file or folder that was not indexed, and why.
ReportRow
One ledger row as the reports count it (0.42.0).
Retrieval
One recorded retrieval.
RetrievalRow
One recorded retrieval, as the access report attaches them.
RowUsage
A row’s usage columns as they are stored. Every figure is optional: a client that logs the model and not the tokens gets the model.
Savings
What the ledger adds up to.
SessionRow
One session’s whole ledger, as the per-session table reads it.
Stamp
What a file looked like when it was read for the store, and the one rule every surface judges “edited since it was indexed” by.
SymbolRow
A symbol row joined with its file’s path — what a graph answer resolves to.
SyncRow
One ledger row as ledger sync sends it: what was retrieved, never the query text, which this struct does not carry.
Unfilled
A ledger row whose usage has not been looked for, as the readers in crate::usage need it to find the model request that made the call.
Unresolved
An edge whose target this store has no definition for.
Writing
Permission to write, given back when it goes out of scope.

Enums§

FileSort
The column a Files listing is ordered by.
LedgerScope
Which rows of the ledger a count is about.

Constants§

AGENT_RETRIEVAL
The rows that are an agent’s retrievals, as SQL: what ledger::figures counts by default. Not a refund (raw-read), not the chain’s own bookkeeping (a note, a refused write, an accept or revoke), and not a person at the portal or the CLI. Every per-store reader below takes it, so a store’s card and its report say what the Ledger page says.
BASIS_BYTES
What the whole-file side of a row was counted from: the file’s bytes.
BASIS_TEXT
What the whole-file side of a row was counted from: the text a reader extracted from it, counted the way the excerpt was.
CODE_CONTEXT
The first format whose code chunks embed the definition they sit inside.
DEFINITION_CHUNKS
The first format whose Markdown chunks are cut at their headings.
FIRST_IMPORT_KEY
Meta key present while a store’s first import is under way (#210).
FORMAT_KEY
FORMAT_VERSION
The store layout this binary understands.
NOTE_TOOL
The tool of a ledger note: a row the chain carries that is not a retrieval, and no total counts.
REFUSED_TOOL
The tool of a call refused because the agent key is read-only; query names the tool. Not a retrieval, so no total counts it either.
SHARDED_FORMAT
The first format that keeps its vectors in shards.
TURBOVEC_V8
The first format whose vector files may be turbovec’s v8.
USAGE_COLUMNS
The usage columns, from column at of a row onwards, in the order USAGE_COLUMNS lists them. None until a reader has looked.

Functions§

accept
acceptance
acceptances
Every acceptance, for counts and for the folders the walk must not prune.
all_hashes
Every stored path’s recorded hash, read once for a whole index pass so its prepare stage can tell an unchanged file without the database.
all_paths
bounded
Run work over conns, interrupting their queries once limit passes.
builtin_targets
How many distinct call and import targets with no definition here are the language’s own builtins or standard library, and so are not in unresolved_targets.
busiest_names
The names with the most edges in or out, busiest first, each counted once on its own index. For the graph page’s choice of where to open: a name with seven hundred definitions is one candidate, not seven hundred.
chunk
chunk_count
Every chunk row, durable or not: what the sidecar can hold a record for.
chunk_ids_of
Drop a file and its chunks, returning the chunk ids so the caller can evict them from the vector index too. The ids of a file’s chunks.
chunking
Which chunking rule this store’s chunks were cut by, in the words stats and the portal print.
chunks_by_month
Chunks by the month their file was indexed, oldest first.
chunks_overlapping
Every chunk of one file whose lines overlap start..=end, in file order.
community_edges
Every calls and imports edge whose target this store defines, up to limit, with the total so a reader is told when the list was cut.
corpus
Measure one store, for the Inside the index page.
coverage_by_language
Per-language coverage for every language the store holds.
delete_file
delete_file_unretired
delete_file without keeping the file’s definitions as history: for a stopped run’s undo, whose files were never the store’s to remember.
delete_graph
Remove one file’s symbols, and with them the edges that leave them.
drop_insert_trigger
Drop the insert trigger an older semlith’s schema puts back when it opens this store, inside the writer’s transaction so it cannot come back before the commit. With it in place every chunk would be indexed twice.
durable_chunks
Chunks whose file has a committed hash — the ones whose vectors are on disk.
edge_counts
How many edges touch name, and how many of them are certain, counted on the indexes alone: (resolved, total).
edges_in
What points at name: callers, importers, references in.
edges_in_first
At most limit of edges_in, the extractor’s certain edges first.
edges_in_resolved
What points at name, each edge resolved against the name’s definitions the way edges_out resolves it from the other end.
edges_out
What the symbols named name point at: callees, imports, references out.
edges_out_short_of_hubs
edges_out without the edges whose target has more than hub definitions, dropped before their candidates are joined.
edges_out_to
edges_out restricted to edges whose target is one of targets, settled exactly as edges_out settles them.
end_first_import
The first import ran to its end: later passes write as any other does.
excerpt_tokens_by_tool
(tool, excerpt_tokens) for every ledger row that names its tool, the newest limit of them: what the Agents page’s typical answer sizes are the median of.
file_count
How many indexed files match groups.
file_facets
file_hash
Content hash recorded for path, if we have indexed it before.
file_rows
The first take matching files in the requested order.
file_stamps
(bytes, indexed_at) for each of paths the store knows.
files_ending_with
Indexed files whose path ends with suffix, most specific first.
files_indexed_since
Files this store read since since, newest first, with when it read them.
files_read_since
Every file read at or after since, newest first, with no cap — the change brief’s “Files re-read” was 200 per store summed (R3).
fill_usage
Write a row’s usage. Every row sharing its query_id is the same call, written once per store that answered, so all of them are filled.
filtered_chunk_ids
Ids of every chunk whose file matches groups.
filtered_image_ids
The image ids whose file matches groups, for a filtered search.
filtered_paths
Indexed paths whose file matches groups, in path order.
first_import
Whether this pass is part of a store’s first import, starting one when the store is empty.
format
Which layout this store’s vectors are in.
free_bytes
Pages SQLite holds free inside the file, in bytes: what a vacuum gives back before anything else is dropped.
get_meta
graph_input
A file’s id and its chunks’ line spans, for writing its graph again.
graph_stats
(symbols, edges) — the graph’s size, for stats and the portal.
grep_cost
The total bytes of every file a grep for name would have to read.
heal_stamps
Bring an unchanged file’s bytes and indexed_at up to what is on disk, when the file’s size or mtime has moved past the row.
held_text
The text the store holds for path, stitched by line from its chunks.
history_bytes_before
Roughly what the definitions retired before before take: their text plus a row’s fixed cost. An estimate for stats, not an accounting — SQLite packs rows into pages and the count cannot see that.
history_count_before
How many definitions were retired before before.
hold_automerge
Stop FTS5 merging segments while a first import writes (#210).
holds_path
Whether this store has path indexed.
image
One image row, joined with its file’s path.
image_count
How many images this store holds.
image_ids
Every image id, for the ids a filtered search may consider.
image_ids_of
The image ids belonging to path, before its row is deleted.
image_rows
Every indexed image, for the decisions table’s “read as image” rows.
insert_chunk
insert_edge
insert_file
insert_image
insert_symbol
is_interrupted
Whether an error is a query bounded interrupted.
is_pinned
Whether this connection is holding a run’s snapshot.
keyword_search
keyword_search_within
The keyword list inside a set of chunk ids, for a filtered search that has the set already.
last_write
The newest indexed_at this store holds, or None when it holds nothing.
ledger_break
Re-walk the chain and return the id of the first row that does not verify.
ledger_clients
How many retrievals each client made, most first.
ledger_facts
Every ledger row with since <= at < until, oldest first.
ledger_head
The hash of the newest ledger row: what --verify checks the chain to.
ledger_keys
The distinct retrievals this store holds, as keys that can be unioned across stores.
ledger_latency
Warm query latency as the ledger recorded it: p50 and p95, in microseconds.
ledger_misses
What the ledger says about the questions semlith did not answer.
ledger_note
Re-anchor the chain after a row that does not verify, by appending a note row onto the current head (0.35.0’s Re-verify with repair).
ledger_repair
Append a note for every unacknowledged break, first to last, and say which rows they were. Bounded, so a chain broken at every row is not a loop.
ledger_retrievals
Every retrieval since since, newest first.
ledger_rows
How many rows the chain holds, notes included: what a verify walked.
ledger_savings
The one savings figure, with the denominators that make it readable.
ledger_savings_since
The savings, over retrievals recorded at or after since.
ledger_sessions
Every session this store recorded, newest last-seen first.
ledger_totals
(queries, clients, excerpt tokens, whole-file tokens) over the whole/// (queries, clients, excerpt tokens, whole-file tokens) over the whole ledger — what the Ledger page shows.
ledger_zero_hit_queries
Queries that found nothing, commonest first.
ledger_zero_hit_queries_since
The questions that found nothing, at or after since.
mark_synced
Record that these rows reached the org’s ledger.
mark_vector_format
Move a format-4 store to TURBOVEC_V8 before this binary writes to it.
matching_files
How many indexed files groups selects.
mtime_ns
A file’s modification time in nanoseconds since the epoch.
names_with_several_definitions
How many names this store defines more than once.
newest_chunk
The highest chunk id, or 0 for none. One step down the rowid b-tree.
open
Open a store’s database, laying down or upgrading its schema first.
owner_of
The type or module a definition’s qualified name puts it in: Fleet for Fleet::search_preferring, store for store::edges_out.
path_bounds
The first and last indexed path in sort order. Their common directory is the common directory of every path, which is the root a report prints paths against when the registry names none (a store opened by --store).
pending_among
Of ids, the chunks whose file is still being embedded, with their text: what a search mid-run embeds on the query path.
pending_share
The share of the store’s chunks still being embedded, or None when nothing is: one indexed probe on every search, and the counts only when the probe finds a run in progress.
pin_snapshot
Hold this reader at the store as it is now, until unpin_snapshot.
prune_history
Drop the definitions retired before before (unix seconds). The retention is the caller’s; see compact.
read_only
Make this connection refuse writes until a writer says otherwise.
readable
(files, chunks, indexed bytes) Whether this store can still be read, as cheaply as that can be asked.
reclaim
Give the database’s dead pages back to the filesystem.
record_counted
record_retrieval, saying what the whole-file side was counted from.
record_legacy_retrieval
Append a row in the shape 0.14.0 wrote, for the test that proves a store holding both kinds still verifies.
record_retrieval
Append one retrieval, chained to the row before it.
record_v1_retrieval
Append a row the way 0.15.0 to 0.41.0 wrote it — chain v1, no basis, no head — for the test that proves a store holding v1 and v2 rows verifies.
refusal
One not-indexed row, by exact path.
refusals
Every not-indexed row, reviewable first, then by class and path.
refuse
Record that path was not indexed. The first time it was seen is kept.
release_automerge
Put back the automerge setting hold_automerge recorded, if any.
report_rows_since
Every ledger row recorded at or after since, oldest first.
retrieval_at
When a ledger row was written, by id.
retrievals
The most recent limit retrievals, newest first.
revoke
salt
The per-store salt fingerprints are keyed with, made on first need.
salt_if_any
The salt, when a store has one. A read-only caller cannot make one, and a store without one has no acceptances to compare against.
secure_delete_fts
Turn on FTS5’s secure delete for chunks_fts, if it is not on: a removed row’s terms then leave the index’s segments at once rather than at their next merge, so a file refused for a secret leaves no searchable term behind. A setting the index keeps; older binaries read it unchanged.
set_file_graph
Record what the parser made of a file: parsed, timeout, or none for a language semlith carries no grammar for.
set_file_units
Record a file’s units (see chunk::COUNTED), by row id or, for a file whose row is already there, by path.
set_meta
set_read_mtime
Record the modification time a file’s read saw. See Stamp::fresh.
set_units_by_path
stats
symbols_by_names
Symbols defined in files matching groups whose name is one of names.
symbols_in_files
Every symbol defined in each of paths.
symbols_named
Every definition of name, across the whole store.
symbols_named_counted
symbols_named with code first — definitions before the headings, keys and selectors that share the name — and how many there are in all.
symbols_past_count
How many definitions this store has retired, over all names.
symbols_past_named
What a symbol used to be, newest first.
symbols_scoped
Symbols to draw for a scope, newest-defined first within each file.
text_of
Everything this store holds for one file, its chunks in order, joined.
uncounted
Stored files of a counted format whose units were never recorded — written by a binary older than 0.34.0. The index pass reads each one again for its count alone.
unfilled
Rows written by an AI client since since (unix seconds) with no usage yet, oldest first. cli and portal rows have no model behind them, and neither has a raw-read row, which the steering hook writes for a read the agent made without semlith.
unpin_snapshot
End pin_snapshot’s read transaction: the reader sees the run’s commit.
unrefuse
Forget that path was not indexed: it has been, or it is gone.
unresolved_out
Targets of name’s edges that the store holds no definition for.
unresolved_targets
Call targets no definition in this store satisfies, commonest first, and how many distinct ones there are.
unsynced
Rows written at or after since and not yet sent, oldest first, at most limit.
unsynced_count
How many rows since since are still to send.
usage_at
usage_rows
Every row with usage, keyed as ledger_keys keys it, for totals per client and per model that count a cross-store call once.
where_defined
Where each of these names is defined, for the hub lines under a community.
write_extraction
Write one file’s extraction: its symbols, then its edges by source name.

Type Aliases§

ChunkSpan
A chunk’s first line, last line and id.
SymbolSpan
One definition’s span and identity: (start, end, name, kind).