Struct Engine 

pub struct Engine { /* private fields */ }

Implementations

impl Engine

pub fn execute(&mut self, sql: &str) -> Result<QueryResult, EngineError>

pub fn execute_with_cancel( &mut self, sql: &str, cancel: CancelToken<'_>, ) -> Result<QueryResult, EngineError>

v4.5 — write path with cooperative cancellation. Same dispatch as execute_in_with_cancel(sql, IMPLICIT_TX, cancel). Kept as a separate entry point for backward-compat with the v4.5 public API.

pub fn execute_in( &mut self, sql: &str, tx_id: TxId, ) -> Result<QueryResult, EngineError>

v4.41.1 multi-slot write entry. Routes sql through the TX slot identified by tx_id so spg-server dispatch can scope each implicit-wrap BEGIN..stmt..COMMIT to its own slot in tx_catalogs. IMPLICIT_TX is the legacy single-slot path every other caller (engine self-tests, replay, spg-embedded) implicitly takes via execute() / execute_with_cancel().

pub fn execute_in_with_cancel( &mut self, sql: &str, tx_id: TxId, cancel: CancelToken<'_>, ) -> Result<QueryResult, EngineError>

v4.41.1 write path with cooperative cancellation + explicit TX scope. Sets self.current_tx for the duration of the call so every exec_* helper transparently sees its TX’s shadow catalog and savepoint stack; restores on exit so the field is only valid mid-call (no leakage across calls).

pub fn prepare(&self, sql: &str) -> Result<Statement, ParseError>

v6.1.1 — parse and pre-process a SQL string ONCE so the resulting Statement can be cached and re-executed via Engine::execute_prepared. Returns the same Statement the simple-query path would synthesise internally (clock rewrites + ORDER BY position-ref resolution applied at prepare time, since both are session-independent). The $N placeholders in the SQL stay as Expr::Placeholder(n) nodes; they’re resolved to concrete values per-call by execute_prepared’s substitution walk.

Pgwire’s Parse (P) message lands here.

pub fn prepare_cached(&mut self, sql: &str) -> Result<Statement, ParseError>

pub fn plan_cache(&self) -> &PlanCache

v6.3.0 — read-only accessor for tests and v6.3.1 invalidation.

pub fn warm_up_plan_cache(&mut self, sqls: &[&str]) -> usize

v7.38 (mailrs prod 7.35 pool-exhaustion incident) — boot-time plan-IR cache warm-up. Walks sqls, calls prepare_cached on each one. Each successful prepare leaves the parsed + reordered + clock-rewritten Statement in the engine-wide plan cache; subsequent Engine::execute / execute_prepared for the same SQL skips parse + JOIN reorder. Returns the count of successfully cached statements.

The mailrs Database::new boot path is the expected caller: pre-warm the top-N query shapes (inbox listing, contacts search, stats) so the first user-facing request doesn’t pay the 2-3 s first-fire cost on the readonly-blocking sqlx pool — which (under prod concurrency) exhausts the pool and stalls the whole UI.

pub fn warm_up_cold_tier(&self) -> usize

v7.38 (mailrs prod 7.35 pool-exhaustion incident) — boot-time cold-tier OS page-cache warm-up. Walks every table in the active catalog, iterates the cold rows via the existing BTree-driven iter_cold_rows_of_table, drops the rows on the floor. The walk’s side effect is that every cold segment file gets mmap-read once — the OS page cache then serves subsequent queries without disk I/O.

Returns the total cold rows touched across all tables. On a hot-only catalog (no cold_segments populated) the call is a near-no-op.

pub fn plan_cache_mut(&mut self) -> &mut PlanCache

v6.3.0 — mutable accessor for v6.3.1 invalidation hooks.

pub fn describe_prepared( &self, stmt: &Statement, ) -> (Vec<u32>, Vec<ColumnSchema>)

v6.3.3 — Describe a prepared Statement without executing. Returns (parameter_oids, output_columns). Empty output_columns means the statement has no row-producing shape we could resolve here — the pgwire layer maps that to NoData.

v7.39 (round 462) — a SELECT over a system catalog view resolves against the same materialised catalog execution builds, so the two paths cannot disagree about what a system view looks like.

pub fn execute_prepared( &mut self, stmt: Statement, params: &[Value<'static>], ) -> Result<QueryResult, EngineError>

v6.1.1 — execute a Statement previously returned by Engine::prepare, substituting Expr::Placeholder(n) nodes for the corresponding Value in params (1-based per PG: $1 → params[0]). Bind-time string parameters are decoded into typed Values by the pgwire layer before this call so the resulting AST hits the same execution path as a simple query — no SQL re-parse.

Pgwire’s Execute (E) message after a Bind (B) lands here.

pub fn execute_prepared_select_no_params( &mut self, stmt: &SelectStatement, cancel: CancelToken<'_>, ) -> Result<QueryResult, EngineError>

v7.37 (SPGS small-query bar) — borrow-based SELECT entry for the pgwire Execute hot path when the portal has no bound parameters. Skips both the AST clone the prepared path used to do at the pgwire call site AND the substitute_ placeholders walk (a no-op when params are empty). Caller must already hold the engine write lock — read would be cleaner, but current_tx mutation keeps it &mut.

pub fn execute_prepared_select_streaming<F>( &mut self, stmt: &SelectStatement, cancel: CancelToken<'_>, emit: F, ) -> Result<usize, EngineError>

where F: FnMut(StreamItem<'_>) -> Result<(), EngineError>,

v7.37 — streaming SELECT for the pgwire Execute hot path. Emits one StreamItem::Header(cols) then one StreamItem::Row(&[&Value]) per surviving row. Returns the total row count for the CommandComplete tag.

For shapes where the engine can stream directly (non-aggregate join projection of bound columns, no ORDER BY / DISTINCT / etc.) no Vec<Row<'static>> is materialised — cell references come straight out of the source tables. For non-streamable shapes the engine runs the full exec_select_cancel, then walks the materialised Vec<Row<'static>> driving the same emit callback (no engine-side win, but pgwire dispatches every Execute through one path).

impl Engine

pub fn execute_prepared_with_cancel( &mut self, stmt: Statement, params: &[Value<'static>], cancel: CancelToken<'_>, ) -> Result<QueryResult, EngineError>

pub fn execute_prepared_in( &mut self, stmt: Statement, params: &[Value<'static>], tx_id: TxId, ) -> Result<QueryResult, EngineError>

v7.39 (round 303, V22) — like Self::execute_prepared_with_cancel but binds the statement to an explicit transaction slot instead of the implicit one. The mysql-wire binary-protocol path uses this so a prepared INSERT/UPDATE lands in the connection’s own BEGIN-opened transaction (and never collides with another connection on slot 0), mirroring what pgwire’s Bind+Execute achieves by rendering bind-final SQL through Self::execute_in.

pub fn execute_prepared_in_with_cancel( &mut self, stmt: Statement, params: &[Value<'static>], tx_id: TxId, cancel: CancelToken<'_>, ) -> Result<QueryResult, EngineError>

impl Engine

pub fn lo_import_bytes( &mut self, want_oid: u32, data: Vec<u8>, ) -> Result<u32, EngineError>

v7.39 (round 249) — resolve the effective COPY FROM target column list, running PG’s pre-file checks in PG’s order: the relation must exist, an explicit column must exist on it, and no column may appear twice — all before a single data row is looked at.

Errors

relation "t" does not exist, column "x" of relation "t" does not exist (42703), column "x" specified more than once (42701). v7.39 (round 343, V40) — store a file the host just read as a large object. The host does the IO (the engine is no_std); the catalog side is the same create_large_object the rest of the lo_* family uses, so an imported object is indistinguishable from one built with lo_from_bytea.

pub fn lo_export_bytes(&self, oid: u32) -> Result<Vec<u8>, EngineError>

v7.39 (round 343, V40) — the bytes the host is about to write out. PG’s message for a missing object, verbatim.

pub fn copy_target_columns( &self, table: &str, columns: Option<&[String]>, ) -> Result<Vec<String>, EngineError>

pub fn copy_from_buffer( &mut self, table: &str, columns: Option<&[String]>, options: &CopyOptions, data: &str, ) -> Result<QueryResult, EngineError>

v7.39 (round 249) — execute a parsed COPY … FROM '<file>' whose file contents the HOST has already read (the engine is no_std and performs no I/O). Lowers to per-row INSERTs via crate::copy::copy_buffer_inserts; outside an explicit transaction the rows are wrapped in one, so a bad row aborts the whole COPY exactly as in PG.

Errors

The failing row’s INSERT error propagates (after rollback).

pub fn copy_to_buffer( &mut self, table: &str, columns: Option<&[String]>, query: Option<&Statement>, options: &CopyOptions, ) -> Result<(String, usize), EngineError>

v7.39 (round 252) — render a COPY … TO '<file>' payload for the HOST to write (the engine is no_std and performs no I/O). Returns the encoded bytes (one line per record, trailing newline) and the DATA row count for the COPY n tag — the HEADER line, when present, is part of the payload but not of the count.

Errors

Same surface as COPY … TO STDOUT (missing relation / column, CSV-mode option refusals).

impl Engine

pub fn compact_cold_segments_with_target( &mut self, target_segment_bytes: u64, ) -> Result<Vec<(String, String, CompactReport)>, EngineError>

v6.7.3 — public shim around Catalog::compact_cold_segments driving every BTree index on every user table. Returns one (table, index, report) triple for each merge that actually happened (no-op (table, index) pairs are filtered out so callers can size persist-side work to the live merges). Caller is responsible for persisting each report.merged_segment_bytes and updating the on-disk segment registry; engine layer is no_std and never touches disk.

Marks every touched table’s cached cold_row_count stale — compaction GC’d some shadowed rows, so the count must be re-derived on the next ANALYZE.

impl Engine

pub fn take_notifications(&mut self) -> Vec<(String, String)>

Drain committed notifications (channel, payload). Wire layers emit each as a NotificationResponse; embedded callers consume directly.

impl Engine

pub const fn publications(&self) -> &Publications

v6.1.2 — read access to the publication catalog. Used by the v6.1.5 publisher-side WAL filter, by SHOW PUBLICATIONS (v6.1.3+), and by e2e tests that need to assert state without going through the wire.

impl Engine

pub fn execute_readonly_on_snapshot( snapshot: &CatalogSnapshot, sql: &str, ) -> Result<QueryResult, EngineError>

v7.11.1 — execute a read-only SQL statement against a CatalogSnapshot without touching this engine. Same semantics as execute_readonly but parameterised on the snapshot’s catalog. Reject DDL/DML the same way execute_readonly does. Static-on-Self so the caller can dispatch without holding an Engine borrow alongside the snapshot.

pub fn execute_readonly_on_snapshot_with_cancel( snapshot: &CatalogSnapshot, sql: &str, cancel: CancelToken<'_>, ) -> Result<QueryResult, EngineError>

v7.11.1 — execute_readonly_on_snapshot with cooperative cancellation. Builds a transient Engine over the snapshot state, runs execute_readonly_with_cancel, drops. The transient engine is cheap to construct (no I/O; everything is just struct moves) and lets the existing read path stay untouched.

pub fn execute_readonly_prepared_on_snapshot( snapshot: &CatalogSnapshot, stmt: Statement, params: &[Value<'static>], ) -> Result<QueryResult, EngineError>

v7.18 — execute a previously-prepared Statement against a CatalogSnapshot in read-only mode. Mirror of Engine::execute_prepared for the fan-out read path: substitutes Expr::Placeholder(n) nodes from params, then dispatches through Engine::execute_readonly_stmt_with_cancel (writes / DDL hit EngineError::WriteRequired). Static-on-Self so multiple readonly threads can dispatch against the same snapshot concurrently without an Engine borrow.

Schema drift contract. The Statement was prepared against some prior catalog. If the snapshot’s catalog has since diverged (DDL renamed / dropped a referenced column / table), execution surfaces the normal EngineError — same shape as PG’s “cached plan must not change result type”. Caller decides whether to re-prepare; engine does NOT auto-retry.

pub fn execute_readonly_prepared_on_snapshot_with_cancel( snapshot: &CatalogSnapshot, stmt: Statement, params: &[Value<'static>], cancel: CancelToken<'_>, ) -> Result<QueryResult, EngineError>

v7.18 — cancellable variant of Engine::execute_readonly_prepared_on_snapshot.

pub fn describe_prepared_on_snapshot( snapshot: &CatalogSnapshot, stmt: &Statement, ) -> (Vec<u32>, Vec<ColumnSchema>)

v7.18 — describe a prepared Statement against a CatalogSnapshot. Same (parameter_oids, output_columns) shape as Engine::describe_prepared; resolves names against the snapshot’s catalog instead of self. Pure function — no engine state read.

pub fn is_readonly_sql(sql: &str) -> bool

v7.18 — does this SQL string classify as read-only? Parses sql with the engine parser and consults Statement::is_readonly(). A parse error returns false (route to the writer path so the user sees the canonical parse error from the writer’s simple-query dispatch). Static-on-Self so the spg-sqlx connection layer can ask without an Engine borrow.

pub fn prepare_on_snapshot( snapshot: &CatalogSnapshot, sql: &str, ) -> Result<Statement, ParseError>

v7.18 — parse + plan a SQL string against a CatalogSnapshot. Mirror of Engine::prepare for the readonly fan-out path: applies the same prepare-time transforms (clock rewrite, GROUP BY ALL expansion, ORDER BY position resolve, cost-based JOIN reorder) but resolves catalog + statistics against the snapshot, not a live engine. Static-on-Self — AsyncReadHandle::prepare calls this without taking the writer lock so multiple read handles can prepare concurrently against frozen views.

Errors

Propagates ParseError from the parser. Schema validation deferred to execute time, same as Engine::prepare.

pub fn execute_readonly(&self, sql: &str) -> Result<QueryResult, EngineError>

v4.0 concurrency: this is the entry point the server takes under an RwLock::read() so multiple SELECT clients run in parallel without serialising on a single mutex.

pub fn prepare_select_streaming( &self, sql: &str, ) -> Result<SelectStatement, EngineError>

v7.37.x (SPGS PROJ wire encode tax) — read-path streaming SELECT. Parses the SQL, applies the same statement-level rewrites the read path does (rewrite_clock_calls, resolve_order_by_position, reorder::reorder_joins), then drives the streaming SELECT executor with the caller’s emit callback. For PROJ-shape SQLs (joined non-aggregate projection of bound columns over thousands of rows) the engine produces each row to the emit fn WITHOUT materialising the result into Vec<Row<'static>> — the per-cell .cloned() and per-row Row::new(values) disappear. On the 25 k-row PROJ shape that’s about 4 ms saved (one less full result allocation pass at the engine output boundary).

Returns the surviving row count emitted (post-WHERE, post-LIMIT) for the CommandComplete tag. Non-SELECT statements surface as Unsupported so the caller can fall back to the materialising read path. v7.37.x (docker-fair SCALARSQ wire-overhead attack) — prepared- SelectStatement variant. Caller has already run parser::parse_statement_with + rewrite_clock_calls + resolve_order_by_position + reorder::reorder_joins (the per-connection parse cache in spg-server’s pgwire layer caches the post-prepare AST and re-applies rewrite_clock_calls per invocation since the clock value embedded in the AST drifts). Otherwise identical to the SQL-string entry point.

pub fn refresh_clock(&self, s: &mut SelectStatement)

Re-apply rewrite_clock_calls to a previously-prepared AST (cache-friendly: the cached AST’s embedded clock literal gets re-pointed to current time without re-parsing).

pub fn execute_readonly_select_prepared( &self, s: &SelectStatement, cancel: CancelToken<'_>, ) -> Result<QueryResult, EngineError>

v7.37.x (docker-fair SCALARSQ wire-overhead attack) — prepared SELECT that returns the full materialised QueryResult instead of driving an emit closure per row. The streaming variant is only a win when the engine can stream rows lazily (joined non-aggregate projection through try_exec_joined_streaming); for shapes that materialise inside the engine anyway (anything with a subquery — including the SCALARSQ shape — and most aggregates), the emit closure dispatch + cell_refs Vec management add ~25-50 µs / 100-row response for zero benefit. This API lets the caller skip the streaming wrapper entirely and iterate the result rows directly into the wire encoder.

pub fn execute_readonly_select_with_arena<'a, F>( &self, s: &SelectStatement, cancel: CancelToken<'_>, arena: &'a Bump, emit: F, ) -> Result<(Vec<ColumnSchema>, usize), EngineError>

where F: FnMut(&[ColumnSchema], &[Value<'a>]) -> Result<(), EngineError>,

v7.37.42-arena Phase 2 — arena-aware streaming SELECT API. On SCALARSQ streaming-shape detection (is_scalarsq_streaming_ shape), routes to exec_scalarsq_streaming and emits each projected row straight out of an arena-backed bumpalo::Vec scratch — no Vec<Row<'static>> ever materialises in the engine for this shape.

Non-streaming shapes fall through to the generic exec_select_cancel materialised path and emit row-by-row off the returned Vec<Row>; callers stay shape-blind.

Caller passes a &'a Bump; per-row projection scratch lives in that arena and drops in O(1) at the caller’s Bump::reset() / scope end. This is the SPG equivalent of PG’s per-query MessageContext / printtup pattern.

The shape check is fast (~10 boolean field reads + items walk); calling on every prepared SELECT is fine.

pub fn execute_readonly_select_streaming_prepared<F>( &self, s: &SelectStatement, cancel: CancelToken<'_>, emit: F, ) -> Result<usize, EngineError>

where F: FnMut(StreamItem<'_>) -> Result<(), EngineError>,

pub fn execute_readonly_select_streaming<F>( &self, sql: &str, cancel: CancelToken<'_>, emit: F, ) -> Result<usize, EngineError>

where F: FnMut(StreamItem<'_>) -> Result<(), EngineError>,

pub fn execute_readonly_with_cancel( &self, sql: &str, cancel: CancelToken<'_>, ) -> Result<QueryResult, EngineError>

v4.5 — read path with cooperative cancellation. Token’s is_cancelled is checked at the start (so a watchdog that already fired returns Cancelled immediately) and at row-loop checkpoints inside exec_select. SHOW paths are O(small) and don’t bother checking.

impl Engine

pub fn set_session_user(&mut self, user: &str)

v7.39 (read01 round 51) — record the connection’s login identity. The server calls this once per connection from the startup packet.

pub fn set_session_authenticated(&mut self)

v7.37 (round 830) — record that this connection’s login identity was verified against a stored credential. The server calls it once per connection, right after set_session_user, when it demanded a password; open-mode connections never do.

pub fn render_style(&self) -> RenderStyle

v7.39 (GUC knife 3) — the parsed session render style (wire / COPY renderers snapshot it once per statement).

pub fn apply_db_role_settings(&mut self, database: &str, role: &str)

v7.39 (round 547) — apply the GUC defaults ALTER ROLE … SET / ALTER DATABASE … SET recorded, in PG’s order of specificity.

Measured on PG18: with all four scopes set, a new session got the role-in-database value. So the least specific is applied first and the most specific last, each overwriting.

pub fn session_tz(&self) -> SessionTz

v7.39 (tz epic) — per-statement session TimeZone snapshot for the timestamptz renderers. SET already validated the value, so an unresolvable name here (host lost its tzdb) degrades to UTC.

pub fn session_param(&self, name: &str) -> Option<&str>

pub fn take_notices(&mut self) -> Vec<Notice>

v7.39 (read01 round 46) — drain the NOTICEs the last statement raised. pgwire emits one NoticeResponse per entry ahead of the statement’s CommandComplete; embedded callers may ignore them.

pub fn session_statement_timeout_ms(&self) -> Option<u64>

v7.37.7 — PG statement_timeout GUC read accessor. Returns the session-set value in milliseconds, parsed from the raw SET statement_timeout = N string. Returns None when:

• the GUC is unset,
• the value is 0 (PG semantics: 0 = no timeout),
• the value fails to parse.

Accepted input shapes mirror PG’s GUC_UNIT_MS parser:

• bare digits: 100 → 100 ms (PG default unit when GUC is in ms)
• explicit ms: 100ms, 100 ms
• seconds: 1s, 30s → 1000 / 30000 ms
• minutes: 5min → 300000 ms

The host (spg-server per-query watchdog) consults this when constructing the CancelToken deadline so a SQL-set SET statement_timeout = 1000 is honoured per-session — the effective deadline becomes min(SPG_QUERY_TIMEOUT_MS, session). Returning None from this fn means “no session override, use the host-level timeout only”.

pub fn session_work_mem_bytes(&self) -> usize

work_mem in BYTES, which is what a sort has to compare against.

The GUC has been accepted, unit-normalised and rendered since round 204, and never read: nothing in the engine turned it into a budget, so a sort’s memory was bounded by the row count and not by the setting. Round 863 added this so the external sort has a ceiling to spill at.

PG’s default is 4 MB, and the same default applies when the session has not set it or the stored value will not parse.

pub fn notice_severity_reaches_client(&self, severity: NoticeSeverity) -> bool

v7.39 (round 621) — does a message of this severity reach the client?

client_min_messages was validated on the way in and then never read, so SET client_min_messages = warning — and even = error — left the NOTICEs coming. Every DROP … IF EXISTS on a name that is not there said so, which is why the standing differential corpus could not use the GUC to quieten its own setup and carried the asymmetry in eighteen of its files.

PG’s order, ascending: debug5 < debug4 < debug3 < debug2 < debug1 < log < notice < warning < error < fatal < panic. A message is sent when its own severity is at least the setting. Anything above warning suppresses both of the severities SPG raises.

impl Engine

pub fn memory_stats(&self) -> MemoryStats

v6.5.0 — materialise spg_stat_segment rows. One row per cold-tier segment with (segment_id, num_rows, num_pages, total_bytes).

v6.7.0 — appended table_name column resolves the v6.5.0 carve-out. Walks every user table’s BTree indices to find which table’s Cold locators point at each segment. Empty string for orphan segments (loaded via SPG_PRELOAD_COLD_SEGMENT before any index registered a locator). The walk is O(tables × indices × keys); cached per call, not across calls — re-walked on every SELECT * FROM spg_stat_segment. v7.31 (memory campaign) — walk the committed catalog and build the per-bucket memory snapshot. O(rows + index entries): operator/monitoring surface, not a query path.

pub const fn with_activity_provider(self, f: ActivityProvider) -> Self

v6.5.2 — register a connection-state provider. spg-server calls this at startup with a function that snapshots its per-pgwire-connection registry. Engine reads through the callback on SELECT * FROM spg_stat_activity.

pub const fn with_audit_providers( self, chain: AuditChainProvider, verify: AuditVerifier, ) -> Self

v6.5.3 — register audit chain provider + verifier.

pub const fn with_slow_query_log( self, threshold_us: u64, logger: SlowQueryLogger, ) -> Self

v6.5.6 — register a slow-query log callback. threshold_us is the floor (in microseconds); only executes above the floor fire the callback. spg-server wires this from SPG_SLOW_QUERY_THRESHOLD_MS (default 100 ms).

pub const fn without_slow_query_log(self) -> Self

v7.37.16 — turn the slow-query log off, the state PG expresses as log_min_duration_statement = -1. Clears the floor and the callback together, so an engine re-registered in the same process cannot inherit a threshold from an earlier boot.

pub fn set_plan_cache_max(&mut self, n: usize)

v6.5.6 — operator knob for plan cache cap. spg-server reads SPG_PLAN_CACHE_MAX env at startup; uses this to override the compile-time default of 256.

pub fn query_stats(&self) -> &QueryStats

v6.5.1 — read-only accessor for tests + v6.5.6 ops resets.

pub fn query_stats_mut(&mut self) -> &mut QueryStats

v6.5.1 — mutable accessor (clear, etc).

pub const fn statistics(&self) -> &Statistics

v6.2.0 — read access to the per-column statistics table. Used by the planner (v6.2.2 selectivity functions read this), by SELECT * FROM spg_statistic, and by e2e tests.

pub fn tables_needing_analyze(&self) -> Vec<String>

v6.2.1 — return tables whose modified-row count crossed the auto-analyze threshold since the last ANALYZE on that table. The threshold is 0.1 × max(row_count, MIN_ROWS_FOR_AUTO_ ANALYZE) — combines PG-style fractional + absolute lower bound so a fresh / tiny table doesn’t get hammered on every INSERT.

Designed to be cheap: walks every user table’s Catalog::table_names() + reads statistics::modified_ since_last_analyze() (BTreeMap lookup). The background worker calls this under engine.read() then drops the lock before re-acquiring engine.write() for the actual ANALYZE.

pub fn autoanalyze_pass(&mut self) -> Result<Vec<String>, EngineError>

v7.37.22 (22.3) — autoanalyze pass.

PG runs autovacuum + autoanalyze on a background timer. SPG’s spg-embedded / spg-server hosts call this from their maintenance loop on a configurable cadence (default 60s, matching PG’s autovacuum_naptime). Each call:

1. Walks tables_needing_analyze() (same threshold as the existing introspection API).
2. Runs ANALYZE <table> on each candidate.
3. Returns the names that were analyzed so the host can log / emit metrics.

Internally identical to ANALYZE name1; ANALYZE name2; … but bundled so the plan-cache invalidation runs once at the end (cheaper than invalidating per-table). The host can call this under the engine write-lock without splicing extra SQL through the parser.

Returns the (possibly empty) list of tables analyzed.

pub fn set_autovacuum(&mut self, on: bool)

v7.37.15 (Phase D) — dead-tuple vacuum pass. The engine-level companion to Self::autoanalyze_pass: physically reclaims committed-tombstoned rows so a gate-on (SPG_MVCC_INPLACE) in-place table’s storage stays bounded.

Under gate-on a DELETE stamps xmax and keeps the row physically present (an UPDATE tombstones the old version and appends the new one); those dead rows accumulate until vacuum removes them. SPG’s xmin / xmax are u64 with no wraparound, so this is pure dead-tuple reclamation — no anti-wraparound freeze is ever needed.

Safety predicate

A tombstoned row (xmax != XMAX_ALIVE) is reclaimed iff its delete-commit version is strictly below oldest_active — the floor of every version any live reader could still resolve as visible (see Self::vacuum_oldest_active). xmax < oldest_active means every current and future snapshot already observes the delete, so no reader can still see the row. When in doubt the row is left in place — never reclaim a row that could still be visible.

Gate-off (default) is a provable no-op

Under the default gate-off path DELETE removes rows physically, so no header ever carries a non-XMAX_ALIVE xmax and there is nothing to reclaim. The explicit guard below returns an empty report without walking any table, so gate-off behaviour is byte-for-byte unchanged.

RowId stability

Reclaiming compacts rows / headers / rowids lock-step (via Table::delete_rows_no_index): every surviving row keeps its stable, never-reused RowId, so held row-locks and tombstone-redo references stay attached to the same row while its physical slot shifts down. Indices are rebuilt against the compacted rows.

Not a daemon (follow-up)

This ships the callable primitive only. Wiring a background thread that calls it on a cadence (PG’s autovacuum_naptime) is a separate concern — a host schedules it under the engine write lock, mirroring how it drives Self::autoanalyze_pass. Noted as a follow-up, not built in this slice.

v7.37.16 — enable/disable the threshold-triggered autovacuum (default ON). Hosts wire SPG_AUTOVACUUM=0|false|off to this.

pub fn set_autovacuum_inline(&mut self, on: bool)

v7.39 (round 173) — turn the statement-exit inline vacuum off. A host that flips this off MUST drive Self::autovacuum_tick from a background worker, or dead rows accumulate without bound (spg-server couples the two: the flag only flips when the worker actually spawns).

pub fn autovacuum_tick(&mut self) -> usize

v7.39 (round 173) — one background-worker autovacuum pass: walk every table, vacuum those over the PG-inspired threshold (dead >= 1000 && dead*4 >= live — same rule as the inline trigger). Returns how many tables were vacuumed. No-op while an explicit transaction is open (its tombstones aren’t committed; the next tick picks the backlog up), when autovacuum is off, or when the in-place gate is off (no tombstones exist).

pub fn vacuum_pass(&mut self, dry_run: bool) -> VacuumReport

dry_run = true counts the reclaimable rows without mutating.

pub fn vacuum_oldest_active(&self) -> u64

v7.37.15 (Phase D) — the conservative vacuum floor: the smallest version any live reader could still resolve as visible. A tombstone with xmax < this is dead to every reader — current and future — so it is safe to reclaim.

Computed as the minimum of:

• current_version() — a fresh reader’s floor: any new snapshot is taken at (or after) the live cursor and sees every delete stamped at or below it, so nothing below the cursor can be resurrected by a future reader;
• min(active_writer_versions) — an in-flight writer reads at its own version and can still see rows deleted after it;
• min(cached RR/SER reader snapshot versions) — a held REPEATABLE READ / SERIALIZABLE snapshot froze its view at capture and can still see rows deleted after that point.

Taking the minimum is deliberately conservative: any live reader drags the floor down, leaving a row that might still be visible in place. Under SPG’s single-global-current_tx serialized model there is at most one in-flight writer, so in the common quiescent case this collapses to current_version().

impl Engine

pub const fn subscriptions(&self) -> &Subscriptions

v6.1.4 — read access to the subscription catalog. Used by the subscription worker (read its own row to find its publications + last applied position), by SHOW SUBSCRIPTIONS, and by e2e tests asserting state directly.

pub fn subscription_advance(&mut self, name: &str, pos: u64) -> bool

v6.1.4 — write access to last_received_pos. Worker calls this after each apply batch (under the engine’s write-lock). Returns false when the subscription was dropped between when the worker received the record and when this call landed.

impl Engine

pub fn create_user( &mut self, name: &str, password: &str, role: Role, salt: [u8; 16], ) -> Result<(), UserError>

salt is supplied by the caller (the host has a random source; the engine is no_std). Caller should pass a fresh 16-byte random value per user.

pub fn drop_user(&mut self, name: &str) -> Result<(), UserError>

pub fn alter_user_password( &mut self, name: &str, password: Option<&str>, ) -> Result<(), UserError>

v7.39 (round 750) — the engine half of ALTER ROLE … PASSWORD: rotate every derived credential form, then re-derive the SCRAM-SHA-256 verifier with a fresh salt (the same source create_user uses). None clears the credential entirely.

pub fn verify_user(&self, name: &str, password: &str) -> Option<Role>

pub fn user_scram(&self, name: &str) -> Option<()>

v7.39 (round 750) — whether a role currently carries a SCRAM verifier (the rotation pins read it; pgwire uses richer paths).

impl Engine

pub fn new() -> Self

pub fn clone_snapshot(&self) -> CatalogSnapshot

v7.11.0 — clone the engine’s committed catalog + read-time state into a frozen CatalogSnapshot. Cheap (Catalog is backed by PersistentVec; cloning is O(log n) per table). Subsequent writes to this engine are invisible to the snapshot; the snapshot is self-contained and can be moved to another thread for concurrent execute_readonly_on_snapshot calls. The basis for [AsyncReadHandle] in spg-embedded-tokio and any other read-fanout pattern.

pub fn role_exists(&self, name: &str) -> bool

v7.39 (round 513) — does this role exist? 'x'::regrole needs it, and roles live on the engine rather than the catalog.

pub fn role_name_for_oid(&self, oid: i64) -> Option<String>

v7.39 (round 520) — the role an oid names, as pg_get_userbyid reports it. The numbering is synth_pg_roles’: base 10, one per user in catalog order.

pub fn current_snapshot(&self) -> Snapshot

v7.37.15 (Phase B / C / E) — current per-row visibility snapshot for in-engine scans. Captures the live writer- version cursor + active-writer set; readers built from this Snapshot see committed state through the moment of capture and DO NOT observe uncommitted writes still inside active_writer_versions.

Phase E: if there’s an explicit transaction in flight under REPEATABLE READ or SERIALIZABLE isolation, returns the snapshot the tx cached at BEGIN time — every statement in the tx sees the same coherent prior-committed view. READ COMMITTED (the default) returns a fresh snapshot per call, matching PG’s per-statement visibility semantics.

oldest_active = version when no writer is in flight (== no dead row could still be observed); else == min of active versions (vacuum-floor).

pub fn begin_writer_version(&mut self) -> u64

v7.37.15 (Phase C) — allocate the next writer version AND add it to the in-flight set so concurrent snapshots hide the resulting writes until Self::commit_writer_version removes the entry. Returns the allocated version so the writer can stamp it on xmin / xmax.

pub fn commit_writer_version(&mut self, v: u64)

v7.37.15 (Phase C) — mark a previously-allocated writer version as committed. Subsequent snapshots stop including it in in_progress, so the writes the version stamped become visible to new readers.

No-op if the version was never allocated; matches PG’s idempotent TransactionIdCommitTree semantics.

pub fn abort_writer_version(&mut self, v: u64)

v7.37.15 (Phase C.2) — mark a previously-allocated writer version as ABORTED (rolled back). Removes it from the in-flight set and records it in aborted_versions so the visibility oracle (Self::xact_status) reports Aborted rather than silently treating it as committed once it leaves the in-flight set. Phase C.3’s in-place write path relies on this: a rolled-back version’s xmin/xmax stamps stay physically present until vacuum reclaims them, and readers must NOT see them.

Idempotent; a no-op if the version was never allocated.

pub fn xact_status(&self, v: u64) -> XactStatus

v7.37.15 (Phase C.2) — the visibility oracle’s terminal-status lookup for one version. In-flight if still allocated, Aborted if it rolled back, otherwise Committed (the default for a version that left the in-flight set the normal way, and for every frozen / pruned old version the engine no longer tracks).

aborted_versions is bounded by pruning below oldest_active during vacuum (Phase D): once no live snapshot can still see an aborted version’s stamps, its entry is dropped. Until Phase D lands the set only grows with rolled-back transactions — noted as a never-die follow-up, not a steady-state leak on the commit path.

pub fn acquire_row_lock( &mut self, rel: RelId, row: RowId, mode: LockMode, version: u64, policy: WaitPolicy, ) -> LockOutcome

v7.37.15 (Phase C.4) — acquire a tuple lock on a stable (RelId, RowId) for writer version. The in-place write path (C.3) calls this before stamping xmax; SELECT ... FOR UPDATE wires here via the parser’s lock-strength clause (C.4). Returns the LockOutcome the caller acts on (grant / park / skip / fail / deadlock-abort).

pub fn release_tx_locks(&mut self, version: u64)

v7.37.15 (Phase C.4) — release every lock + wait held by version at transaction end. Called from exec_commit / exec_rollback alongside the writer-version bookkeeping.

pub fn locked_row_count(&self) -> usize

v7.37.15 (Phase C.4) — number of rows currently locked, for the pg_locks enumeration and tests.

pub fn mvcc_inplace(&self) -> bool

v7.37.15 (Phase C.3) — is the in-place MVCC write path enabled? false (default) keeps legacy physical DELETE/UPDATE. The C.3 writers consult this to choose tombstone-vs-physical.

pub fn set_mvcc_inplace(&mut self, on: bool)

v7.37.15 (Phase C.3) — enable/disable the in-place MVCC write path. Called by the host after reading SPG_MVCC_INPLACE (the no_std engine can’t read the environment itself). Off until the write path is proven against PG18 differential tests.

pub fn set_backend_count_fn(&mut self, f: BackendCountFn)

v7.39 (parallel-agg P0) — inject the host’s parallel executor (see ParallelRunner). Called once at host startup; the engine stays single-threaded without it. v7.39 (pg_stat knife A) — inject the host’s live backend count.

pub fn set_wal_lsn_fn(&mut self, f: WalLsnFn)

v7.39 (read01 pgstatfuncs.c) — inject the host’s calling-connection identity for pg_backend_pid(). v7.39 (round 476) — register the WAL byte-position provider.

pub fn set_backend_pid_fn(&mut self, f: BackendPidFn)

pub fn set_backend_signal_fn(&mut self, f: BackendSignalFn)

v7.39 (round 318, V51) — inject the host’s connection-control hook, so pg_cancel_backend / pg_terminate_backend / KILL act instead of answering a constant.

pub fn set_temp_run_factory(&mut self, f: TempRunFactory)

v7.39 (round 786, T35 Phase A) — install the host’s spill-run factory. Without one the engine cannot spill and a sort that outgrows max_query_bytes keeps refusing, which is exactly the behaviour every caller has today.

pub fn can_spill(&self) -> bool

Whether spilling is available in this process.

pub fn set_tz_fns( &mut self, offset: TzOffsetFn, localize: TzLocalizeFn, canon: TzCanonFn, abbrev: TzAbbrevFn, )

v7.39 (tz epic) — inject the host’s IANA timezone lookups (spg-tzif’s fn family on std hosts).

pub fn set_tz_all_fn(&mut self, all: TzAllFn)

v7.39 (round 502) — the zone enumerator behind pg_timezone_names. Separate from set_tz_fns so an embedder that already calls that one keeps compiling.

pub fn set_parallel_runner(&mut self, runner: Arc<dyn ParallelRunner>)

pub fn next_writer_version(&self) -> u64

v7.37.15 (Phase C) — allocate a fresh version number for the next write. Always strictly monotonic + process-wide shared so concurrent engines on the same process agree on “tx 17 commits before tx 18”. Phase C writer paths call this once per INSERT / UPDATE / DELETE statement to obtain the version they’ll stamp on the new row’s xmin (or the existing row’s xmax).

Returns XMIN_FROZEN when MVCC stamping is intentionally off (legacy in_memory flow / WAL replay): the writer then takes the legacy frozen-insert short-circuit path inside Table::insert_with_xmin.

pub fn writer_version_for_current_stmt(&mut self) -> u64

v7.37.15 (Phase C) — version a writer should stamp on rows produced by the current statement. Inside an explicit transaction the version is the tx’s pre-allocated one (so every statement in the tx commits atomically at COMMIT); in autocommit it allocates a fresh version per statement.

This is the canonical helper engine writers should call — using it instead of next_writer_version ensures explicit-tx semantics where every row produced by the tx shares one xmin and concurrent readers don’t see partial state until COMMIT.

v7.37.15 (Epic W slice 2) — takes &mut self so the autocommit branch can memoize its freshly-minted version in stmt_writer_version. next_writer_version() is a fetch_add, so without memoization a second call within one statement (the redo drain post-stamps the captured RowChanges) would allocate a different version than the writes used. Memoizing makes the value stable for the statement’s lifetime; it is reset per execute_in_with_cancel, so the counter still advances exactly once per autocommit statement — identical to before.

pub fn restore(catalog: Catalog) -> Self

Construct an engine restored from a previously-snapshotted catalog (see snapshot()).

pub fn restore_envelope(buf: &[u8]) -> Result<Self, EngineError>

Restore an engine + user table from a v4.1 envelope produced by snapshot_with_users(). Falls back to plain catalog-only restore if the envelope magic isn’t present (so v3.x snapshot files still load). v6.1.2 adds the optional publications trailer (envelope v3); a v1/v2 envelope deserialises to an empty publication table.

pub const fn users(&self) -> &UserStore

pub fn set_current_session(&mut self, id: u32)

Builder: attach a wall clock so NOW() / CURRENT_TIMESTAMP / CURRENT_DATE evaluate to a real value instead of erroring out. v7.39 (round 279) — announce which connection is about to run. The server calls this before every statement; embedded hosts never do and stay on session 0.

Swapping parks the outgoing connection’s state and installs the incoming one’s, creating it on first sight. The plan cache is cleared because the string-literal dialect is part of what swaps and the same SQL text lexes differently under it.

pub fn end_session(&mut self, id: u32)

v7.39 (round 279) — a connection has gone away: drop its parked state and release every advisory lock it still held, which is what PG does at backend exit.

pub fn set_backslash_escapes(&mut self, flag: bool)

v7.39 (round 302, V15) — force the current session’s string-literal dialect. A MySQL-protocol connection defaults to MySQL semantics (backslash is an escape: '\n' is a newline), which PG’s own default (standard_conforming_strings = on) does not do. The mysql-wire shim calls this once, right after installing its session, so a client that never sends SET sql_mode still gets MySQL string handling; a later SET sql_mode='NO_BACKSLASH_ESCAPES' flips it back through the normal SET path. Clearing the plan cache mirrors [set_current_session] — the same SQL text lexes differently once the flag moves.

pub const fn with_clock(self, clock: ClockFn) -> Self

pub const fn with_salt_fn(self, f: SaltFn) -> Self

Builder: attach an OS-backed RNG for per-user password salts. The host (spg-server) typically wires this to /dev/urandom.

pub fn with_env_cfg(self, env_cfg: EnvConfig) -> Self

v7.38 元机制 D — install a frozen testkit::EnvConfig snapshot.

Hosts (spg-server, spg-embedded, tests) call this once at engine init with either EnvConfig::from_env() (production-with-test-vars) or EnvConfig::builder()....build() (programmatic). After construction the engine never reads env vars; all test-mode behaviour flows through self.env_cfg().

pub fn env_cfg(&self) -> &EnvConfig

v7.38 元机制 D — frozen test-mode GUC snapshot. Hot paths gate nondeterministic surfaces on fields of this struct; production default keeps every field at false / None / Auto so the optimiser can const-fold the gate.

pub fn rng_seed(&self) -> u64

v7.38 元机制 D acceptor — single seed source for every nondeterministic engine subsystem (hash builders, randomised tie-breakers, …). Honour SPG_TEST_RANDOM_SEED=N when set; otherwise derive from the engine’s wall clock (production) or fall back to a fixed sentinel when the host hasn’t installed a clock. Two engines built with the same builder seed return byte-equal output for the same query. See xtests/sigil/test-mode-gucs.md.

pub fn enter_injection_scope(&self) -> InjectionGuard

v7.38 P0 元机制 A — push this engine’s InjectionStore onto the thread-local stack so any injection_point!() reached during the returned guard’s lifetime resolves against this engine. Mirrors PG’s per-backend injection table.

Returns a no-op guard when the injection-points feature is off so call sites don’t need #[cfg].

pub const fn with_max_query_rows(self, n: usize) -> Self

Builder: cap the number of rows a single SELECT may return. Exceeding the cap raises EngineError::RowLimitExceeded — the bound is checked inside the executor so a runaway catalog scan can’t allocate millions of rows before the server gets a chance to reject the result.

pub const fn with_max_query_bytes(self, n: usize) -> Self

Builder: cap the approximate heap bytes a single SELECT’s join/filter materialisation may hold. Exceeding the cap raises EngineError::QueryBytesExceeded. Rows are the wrong unit when one row carries a multi-MB body (mailrs round-26: 1000-row batches of full mail text walked a 15 GiB host into reclaim livelock without ever tripping a row ceiling).

pub const fn catalog(&self) -> &Catalog

The committed catalog. Note: during a transaction this returns the pre-TX state — SELECT inside a TX goes through execute() and reads the shadow. Tests that inspect outside-TX state should use this.

pub fn snapshot_data(&self) -> EngineSnapshot

Capture a frozen view of the committed engine state. Catalog is O(1) Arc bump; trailers are cheap clones. Decouples “capture” (needs &Engine) from “serialize” (CPU, no engine access) — the seam the background-checkpoint worker rides in CoW-2.

pub fn snapshot(&self) -> Vec<u8>

Serialize the committed catalog to bytes. v0.6 was full-snapshot; v0.9 adds the rule that an open TX’s shadow is never snapshotted — only the post-COMMIT state is persisted. v4.1 wraps the catalog in an envelope when there are users to persist; an empty user table snapshots as the bare catalog format (backwards-compat with v3.x readers). v6.1.2 adds publications to the envelope condition: either non-empty users OR non-empty publications now triggers the envelope path.

pub fn in_transaction(&self) -> bool

True when at least one TX slot is in flight. v4.41.1 runtime invariant: at most one slot active at a time (dispatch holds engine.write() across the entire wrap). v4.42 will let this return true with multiple slots concurrently.

pub fn is_tx_open(&self, tx_id: TxId) -> bool

v7.37 C.5 (A.2) — per-connection in-transaction test. A given connection is “in a transaction” iff its own tx_id has an open shadow slot. Unlike [in_transaction] (which is true if any tx is open), this lets concurrent connections each carry their own explicit transaction without colliding on the global slot. IMPLICIT_TX never has a persistent slot (autocommit reads/writes the main catalog), so this is false for the autocommit id.

pub fn alloc_tx_id(&mut self) -> TxId

v4.41.1 allocate a fresh TX handle. Used by spg-server dispatch to scope each implicit-wrap BEGIN..stmt..COMMIT to its own slot in tx_catalogs. v4.42 — the commit-barrier leader allocates one of these per task in its group, runs BEGIN+sql+COMMIT sequentially under a single engine.write() so each task’s mutations accumulate into shared state, then either keeps the accumulated state (fsync OK) or restores the pre-image via replace_catalog (fsync err).

pub fn replace_catalog(&mut self, catalog: Catalog)

v4.42 — atomically replace the live catalog. Used by the commit-barrier leader to roll back a group whose batched fsync failed: the leader snapshots engine.catalog().clone() (O(1) Arc bump after the v4.39/v4.40 persistent migration) at group start, sequentially applies each task’s BEGIN+sql+ COMMIT under the same write lock to accumulate mutations into shared state, batches the WAL bytes, fsyncs once, and on failure calls this with the pre-image to undo every task in the group at once.

Does NOT touch tx_catalogs / current_tx. Any explicit-TX slot from a concurrent client (created via the legacy IMPLICIT_TX-less dispatch path or via the future MVCC-readers v5+ work) has its own snapshot baked into the slot — restoring self.catalog to the pre-image leaves those slots untouched, exactly as they were when the leader took the lock. The leader’s own implicit-TX slots are all already discarded (exec_commit removed them as each task’s COMMIT ran) by the time this is reached.

pub fn freeze_oldest_to_cold( &mut self, table_name: &str, index_name: &str, max_rows: usize, ) -> Result<FreezeReport, EngineError>

v6.7.0 — public shim around Catalog::freeze_oldest_to_cold so tests + the spg-server freezer can drive a freeze without reaching into the private active_catalog_mut. v6.7.4 parallel freezer will build on this surface.

Marks the table’s cached cold_row_count stale because the freeze added cold locators that ANALYZE hasn’t yet refreshed.

pub fn receive_cold_segment( &mut self, segment_id: u32, bytes: Vec<u8>, ) -> Result<(), EngineError>

v6.7.5 — public shim used by the spg-server follower’s segment-forwarding receiver. Registers a cold-tier segment at a specific id (the master’s id, as transmitted on the wire) so the follower’s BTree-Cold locators stay byte- identical with the master’s. Wraps Catalog::load_segment_bytes_at under the standard clone-mutate-replace pattern.

Returns Ok(()) on success and on the “slot already occupied” case — a follower mid-reconnect may receive a segment chunk for a segment_id it already has on disk (forwarded last session); the caller should treat that path as a no-op rather than a fatal error.

pub fn set_redo_capture(&mut self, on: bool)

v7.34 (crash-recovery P0 #2) — turn row-level redo capture on/off. The embedding layer enables it when persistence is on so each mutating execute records the physical RowChanges it applied (drained via Engine::take_redo). Off = zero capture overhead.

pub fn redo_capture_enabled(&self) -> bool

v7.37.8 — read accessor for tests / observability. The embedding layer flips this on once per open_path (after replay completes) when SPG_WAL_ROW_REDO is enabled (now default in v7.37.8). A consumer that wants to verify the post-upgrade contract (“writes go to V5 ROW_REDO by default”) reads this through Database::engine_redo_capture() instead of inspecting WAL bytes (which the auto-checkpoint truncates on Drop).

pub fn current_isolation_level(&self) -> IsolationLevel

v7.38 轴 4 — currently-selected SQL isolation level. Default ReadCommitted after construction; updated by SET TRANSACTION ISOLATION LEVEL …. Read by SHOW transaction_isolation and any future MVCC/SSI gate.

pub fn take_redo(&mut self) -> Vec<RowChange>

v7.34 — take the redo captured by the most recent successful mutating execute (empty when capture is off, the statement was a read, or it changed nothing). The embedding layer writes these to the WAL in place of the SQL text.

pub fn apply_redo(&mut self, changes: &[RowChange]) -> Result<(), EngineError>

v7.34 (crash-recovery P0 #2) — replay a row-level redo log onto the committed catalog (the row-level WAL recovery primitive: apply the captured physical changes from a checkpoint baseline, in place of re-executing the SQL). Trusts the log — no uniqueness/FK/parse.

Trait Implementations

impl Debug for Engine

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Engine

fn default() -> Engine

Returns the “default value” for a type.

impl XactStatusOracle for Engine

v7.37.15 (Phase C.2) — the engine is its own visibility oracle. Scans hold &Engine while reading, so a scan site can pass self as the XactStatusOracle alongside its Snapshot when the visibility gate migrates from visible to visible_with_status (next Phase C step). Delegates to Engine::xact_status.

fn status(&self, version: u64) -> XactStatus

Terminal status of version. Implementations return XactStatus::Committed for any version they no longer track (pruned below oldest_active, or frozen) — those are, by definition, committed-and-old.