Expand description
User-visible catalog of data tables in the workspace.
Tracks, for every user-facing table:
| Field | Who populates |
|---|---|
table_name | MCP (stub) |
load_tool | MCP (stub) |
load_params | MCP (stub) |
loaded_at | MCP (stub) |
last_refreshed_at | MCP (stub, bumped on every explicit load) |
row_count | MCP (stub, refreshed opportunistically) |
source_url | User / LLM via set_table_metadata |
source_description | User / LLM |
purpose | User / LLM |
license | User / LLM |
notes | User / LLM |
data_url | User / LLM via set_table_metadata |
The backing table is _table_catalog. Each writable database that
receives an ingest gets its own catalog, lazily seeded on first
ingest (or first set_table_metadata):
- Ephemeral primary writes stub into the persistent catalog (so a long-running session’s bookkeeping survives the ephemeral file’s per-session deletion). Reconciliation cleans up rows whose tables no longer exist on engine bootstrap.
database = "persistent"writes to the persistent catalog directly (same destination as the ephemeral case).database = "<user-attached-writable>"writes to that DB’s own_table_catalog. Read-only attachments never get a catalog.--ephemeral-onlymode (no persistent attached) is a degraded mode where catalog operations are no-ops because there is nowhere durable to put bookkeeping.
Per-DB catalog routing is exposed via the *_in variants
(ensure_exists_in, upsert_stub_in, set_metadata_in,
get_in, list_in, delete_for_in, reconcile_in). The
original names (ensure_exists, upsert_stub, …) become
1-line wrappers passing target_db = None (which resolves to the
persistent catalog).
_table_catalog is hidden from the user-visible
Engine::describe_tables output (see
crate::engine::is_internal_table) so the LLM doesn’t see it
alongside its data tables; users who want to inspect it directly
can run SELECT * FROM "<db>"."public"."_table_catalog".
§Concurrency
Catalog upserts use an optimistic UPDATE-then-conditional-INSERT
pattern. Hyper rejects PRIMARY KEY (“Index support is disabled”)
and INSERT … ON CONFLICT (“syntax error: got ON”), but supports
INSERT … SELECT … WHERE NOT EXISTS (…) as a single atomic
statement. Because hyperd serializes individual statements,
the conditional INSERT can never produce duplicate rows — even
when multiple MCP server processes race to upsert the same
table_name concurrently. The UPDATE uses last-writer-wins
semantics for mechanical fields while preserving prose columns
untouched.
Structs§
- Catalog
Entry - One row in
TABLE_CATALOG_TABLE. - Metadata
Fields - Partial update payload for
set_metadata.Nonemeans “leave the existing value alone”;Some(String::new())clears the field.
Constants§
- TABLE_
CATALOG_ TABLE - Backing table name. Lives in the persistent attachment under
"persistent"."public"."_table_catalog". Hidden fromEngine::describe_tables; users who want to inspect it directly can run a fully-qualified SELECT.
Functions§
- delete_
for - Backward-compatible wrapper: delete from the persistent catalog.
- delete_
for_ in - Delete the catalog row (if any) for
table_nameintarget_db. Idempotent — no error if the row doesn’t exist. - ensure_
exists - Backward-compatible wrapper: ensure the catalog exists in the
persistent attachment (or no-op in
--ephemeral-onlymode). - ensure_
exists_ in - Idempotently create the backing table in
target_db. Safe to call on every engine init or every catalog write — if the table already exists this is a no-op. The schema is the only one the code targets; all prose columns are nullable, so a plainNULLinsert is always well-formed. - ensure_
exists_ in_ database Deprecated - Backward-compatible wrapper for the old
ensure_exists_in_databasename. Delegates toensure_exists_in. - get
- Backward-compatible wrapper: read from the persistent catalog.
- get_in
- Fetch a single row from
target_dbbytable_name, orOk(None)if absent. - list
- Backward-compatible wrapper: list rows from the persistent catalog.
- list_in
- Fetch every catalog row from
target_dbin name-sorted order. Returns an emptyVecwhen no catalog exists in the target. - reconcile
- Backward-compatible wrapper: reconcile the persistent catalog.
- reconcile_
in - Synchronize the catalog in
target_dbagainst the current set of user tables in that DB. - set_
metadata - Backward-compatible wrapper: update prose fields in the persistent catalog.
- set_
metadata_ in - Partial UPDATE of prose fields for one table in
target_db. Errors withErrorCode::TableNotFoundif there is no catalog row fortable_namein that DB. - upsert_
stub - Backward-compatible wrapper: upsert into the persistent catalog.
- upsert_
stub_ in - Upsert a catalog row for
table_nameintarget_db, carrying forward prose fields from any existing row and refreshing mechanical fields.