Crate spg_engine 

SPG execution engine — v0.3 wires the SQL front-end to the in-memory storage layer. Implements CREATE TABLE, single-row INSERT VALUES, and SELECT * FROM <table> (no WHERE yet — that lands in v0.4 alongside expression evaluation against rows).

Re-exports

pub use crate::users::Role;

pub use crate::users::ScramSecrets;

pub use crate::users::UserError;

pub use crate::users::UserStore;

pub use subquery::BATCHED_SCALAR_FALL_THROUGH_COUNT;

pub use subquery::BATCHED_SCALAR_KEYED_FIRE_COUNT;

pub use subquery::BATCHED_SCALAR_KEYED_PROBE_COUNT;

pub use subquery::EXISTS_BATCH_FALL_THROUGH_COUNT;

pub use subquery::EXISTS_BATCH_FIRE_COUNT;

pub use subquery::EXISTS_PULLUP_BAIL_INNER_FROM;

pub use subquery::EXISTS_PULLUP_BAIL_INNER_SHAPE;

pub use subquery::EXISTS_PULLUP_BAIL_MULTICOL_DISABLED;

pub use subquery::EXISTS_PULLUP_BAIL_NO_CORR;

pub use subquery::EXISTS_PULLUP_BAIL_NO_WHERE;

pub use subquery::EXISTS_PULLUP_BAIL_RESIDUAL_NOT_INNER;

pub use subquery::EXISTS_PULLUP_BAIL_UNIQUE_KEY_MISSING;

pub use subquery::EXISTS_PULLUP_CANDIDATE_COUNT;

pub use subquery::EXISTS_PULLUP_FIRE_COUNT;

pub use subquery::EXISTS_PULLUP_MULTICOL_DISABLE;

pub use subquery::PULLUP_LIMIT1_FIRE_COUNT;

pub use subquery::SCALARSQ_PK_PROBE_FIRED;

pub use subquery::ScalarPkProbeFastPath;

pub use subquery::expr_tree_has_subquery;

pub use tempstore::SpillStats;

pub use tempstore::TempRun;

pub use tempstore::TempRunFactory;

pub use tempstore::TempStoreError;

Modules

aggregate

Aggregate executor.

copy

v7.22 (mailrs round-13 / T2) — shared COPY text-format helpers.

describe

v6.3.3 — Describe statement pre-Execute.

eval

Expression evaluator. Given a parsed Expr, a Row, and the row’s column schema, produce a Value. v0.4 implements:

fts

v7.12.1 — full-text search lexer / stemmer.

json

v4.14 minimal JSON parser for the -> / ->> operators.

largeobject

v7.39 (round 342+, V40) — the two large-object calls that touch a SERVER FILE.

locks

v7.37.15 (Phase C.4) — row-level lock table, the four PG tuple-lock modes, and wait-for deadlock detection.

memoize

v6.2.6 — Memoize cache for correlated subqueries.

plan_cache

v6.3.0 — Engine-level plan cache.

publications

v6.1.2 — logical-replication publication catalog.

query_stats

v6.5.1 — per-distinct-SQL LRU stat collector.

reorder

v6.2.3 — JOIN reorder planner pass.

scalarsq_streaming

selectivity

v6.2.2 — selectivity estimation over per-column statistics.

statistics

v6.2.0 — per-column statistics for the cost-based optimizer.

subquery

Correlated-subquery evaluation split out of lib.rs (lib.rs split 5): the per-row eval_expr_with_correlated path (clones the expression, substitutes outer-row columns into each surviving subquery node, runs the inner SELECT, folds the literal result back) plus the subquery_replacement pre-walk that materialises uncorrelated subquery nodes once, and the try_batch_correlated_scalar keyed-probe optimisation (round-22 phase 3) that runs a correlated scalar subquery ONCE without the correlation and folds rows into a key→value map. impl Engine methods; the bare-SELECT / DML / join row loops drive eval_expr_with_correlated, and select.rs drives subquery_replacement / try_batch_correlated_scalar.

subscriptions

v6.1.4 — logical-replication subscription catalog.

tempstore

v7.39 (round 786, T35 Phase A) — host-provided temporary storage for query-time spilling.

testkit

Test-mode GUC framework (v7.38 P0 元机制 D).

triggers

v7.12.4 — PL/pgSQL row-level trigger executor.

users

User table + RBAC types for v4.1.

Macros

bump_counter

injection_point

Hook a perf-irrelevant test-only injection point into a code path.

Structs

ActivityRow

v6.5.2 — one row of spg_stat_activity. Engine-public so spg-server can construct rows without re-exporting internal dispatch types.

AllCommitted

An oracle that reports every version as committed. Equivalent to the pre-Phase-C.2 two-state world; lets a caller that does not yet track aborts reuse Snapshot::visible_with_status and get behaviour identical to Snapshot::visible.

AuditRow

v6.5.3 — one row of spg_audit_chain. Engine-public so spg-server can construct rows directly from AuditEntry.

CancelToken

CatalogSnapshot

v7.11.0 — frozen read-only view of the engine’s committed state. Constructed via Engine::clone_snapshot. Holds clones of the catalog, statistics, clock function, and row-cap config — the four fields the execute_readonly path actually reads. Cheap to Clone (each clone shares the underlying PersistentVec row storage; only the trie root pointers copy). Send + Sync so a snapshot can be moved across tokio::task::spawn_blocking boundaries without coordination.

Engine

EngineSnapshot

Clone is O(1) on the catalog (Arc bump) and cheap typed-clones on the trailers. Decouples “capture state” from “serialize bytes” so the background-checkpoint worker can hold the snapshot and produce bytes off the engine write lock.

InProgressSet

Compact in-progress set. Stored as a sorted Vec<u64> so the contains check is a binary search — O(log n) and zero allocation per lookup. We expect n to be tens at most (the active transaction count); for that range bsearch beats a hashset by a wide margin on both wall-clock and cache.

MemoryStats

v7.31 — whole-engine memory snapshot: the polling form of the round-26 ask-4 watermark signal. Hosts compare total_approx_resident_bytes (+ their own WAL/file accounting) against their deployment ceiling and shed/shrink before the kernel does it for them.

Notice

v7.39 (round 318, V41) — one diagnostic the statement raised, in PG’s exact wording minus the severity banner (the wire layer adds that).

ParallelRunnerSlot

Engine slot for the injected runner — a newtype so the Engine derive(Debug) keeps working over the non-Debug trait object.

RowHeader

Per-row MVCC visibility header.

SelectStatement

Snapshot

Per-statement / per-transaction snapshot.

TableMemoryStats

v7.31 (memory campaign — ceiling-first / never-die, design v1) — per-table slice of the engine’s resident-memory accounting. hot_encoded_bytes is the storage layer’s maintained meter (what the rows encode to); approx_resident_bytes is what they COST in RAM (per-cell enum slots + heap payloads via approx_row_bytes) — the gap between the two is the representation multiplier the round-26 report measured at ~11× end-to-end.

TxId

v4.5 cooperative cancellation token. A long-running SELECT / UPDATE / DELETE checks is_cancelled at row-loop checkpoints and bails with EngineError::Cancelled. The host (spg-server) creates an AtomicBool per query, spawns a watchdog thread that sets it after SPG_QUERY_TIMEOUT_MS, and passes it via execute_with_cancel / execute_readonly_with_cancel.

Enums

EngineError

All errors the engine can return.

NoticeSeverity

v7.39 (round 318, V41) — how loud a diagnostic the statement raised is. PG distinguishes them on the wire (S/V fields of NoticeResponse) and clients act on it: psql prints WARNING: in a different colour, and several drivers surface warnings to the application while dropping notices. Emitting everything as NOTICE loses that.

ParsedStatement

QueryResult

Result of executing one statement.

RowCells

One row’s cells, in whichever shape the producer already holds them.

RowChange

In-memory table: schema + a persistent row vector + secondary indices.

SessionTz

v7.39 (tz epic) — per-statement snapshot of the session TimeZone, consumed per-VALUE by the timestamptz renderers (a DST zone’s offset depends on the instant being rendered).

StreamItem

v7.37 — one item in the streaming SELECT emit channel. The engine yields exactly one Header (before any row) then zero or more Rows. Pgwire (or any other consumer) decides how to turn those into wire bytes.

XactStatus

v7.37.15 (Phase C.2) — terminal state of a transaction / row version, as seen by the visibility oracle.

Constants

COMPACTION_TARGET_DEFAULT_BYTES

v6.7.3 — default segment-size threshold used by COMPACT COLD SEGMENTS when no explicit target is supplied. Segments whose OwnedSegment::bytes().len() is strictly less than this value are eligible to merge. spg-server reads SPG_COMPACTION_TARGET_SEGMENT_BYTES to override.

IMPLICIT_TX

Reserved slot used by Engine::execute(sql) — the legacy single- global-shadow path. New alloc_tx_id handles start at 1.

MUTATING_CALL_NEEDLES

v7.39 (round 498) — the function names that must reach the &mut executor, as lowercase byte needles including the opening paren.

XMAX_ALIVE

Sentinel value used for xmax when the row has NOT been deleted. PG uses InvalidTransactionId = 0; SPG matches.

XMIN_FROZEN

Sentinel used for xmin on rows loaded from a pre-v7.37.15 envelope. Any non-zero value < every real transaction id works — we pick 1 (PG uses FrozenTransactionId = 2).

Statics

ANTI_JOIN_FAST_PATH_FIRED

ANTI_JOIN_FAST_PATH_TRIED

DISTINCT_DUP_DROPPED

MATVIEW_DELTA_APPLIED

MATVIEW_DELTA_BAILED

MATVIEW_FANOUT_BUFFERED

v7.39 — diagnostic counter: how many aggregate scans took the sharded path (read by benches to ground-truth activation). v7.39 (round 740) — matview delta ground-truth counters (the r735 lesson: a green content pin cannot distinguish “delta applied” from “silently fell back to full”; these can).

PARALLEL_AGG_FIRED

PROJ_DIRECT_FIRE

PROJ_ROW_BUILT

SCAN_PATH_ENTERED

v7.39 (round 485) — how many projected rows the single-table scan builds, and how many of those the DISTINCT probe throws away again.

UNIQ_PROBE_CALLS

v7.39 (round 166) — probe idx for a live row whose collated key equals key (the fold of the row being written). Returns the row position of the first conflicting live row. fold recomputes the collated key of a candidate row so collation / bpchar semantics stay byte-identical with the HashSet path; tombstoned rows are skipped the same way; Cold locators are skipped because the fold path only ever scanned hot rows. v7.39 (round 492) — how many locators the uniqueness probe walks, and how many probes there are.

UNIQ_PROBE_LOCATORS

Traits

ParallelRunner

v7.39 (parallel-agg P0) — host-injected parallel executor. The engine is no_std and cannot spawn threads; like ClockFn / RandomFn, the std-side host (spg-server / embedded-tokio) injects an implementation at startup. None (the default, and the only option in pure-no_std embeddings) keeps every code path single-threaded and byte-identical to pre-P0 behaviour.

XactStatusOracle

v7.37.15 (Phase C.2) — the visibility oracle: maps a version id to its terminal XactStatus.

Functions

format_bigint_2d_text_pub

format_bit_string

v7.37.5 ζ-A — render a BIT / BIT VARYING as a binary string of '0' and '1' chars (PG canonical text form). Bytes are packed big-endian within each byte: the most-significant bit of byte 0 is bit 0 of the bit string.

format_circle

v7.37.5 ε — render a Circle as PG canonical <(x,y),r>.

format_hstore_text

v7.17.0 Phase 3.P0-39 — pub re-export so pgwire + sqllogictest share the single hstore renderer.

format_inet

format_int_2d_text_pub

v7.17.0 Phase 3.P0-40 — pub re-exports so pgwire + sqllogictest share the single 2D-array renderer.

format_line

v7.37.5 ε — render a Line as PG canonical {a,b,c} (Ax+By+C=0).

format_lseg

v7.37.5 ε — render an Lseg as PG canonical [(x1,y1),(x2,y2)].

format_macaddr

v7.37.5 ζ-A — render a MACADDR (6 bytes) as aa:bb:cc:dd:ee:ff.

format_macaddr8

v7.37.5 ζ-A — render a MACADDR8 (8 bytes) as aa:bb:cc:dd:ee:ff:00:11.

format_multirange

v7.37.5 δ — render a Multirange in PG external form {[a,b),[c,d)}. Empty multirange renders as {}. Each range element is formatted with the same [/(/]/) bracket grammar as scalar Value::Range. RangeSpan carries no kind (it lives on the parent Multirange), so this routes element formatting through format_range_element as Value::Range does.

format_path

v7.37.5 ε — render a Path as PG canonical [(x,y),...] open or ((x,y),...) closed.

format_pg_box

v7.37.5 ε — render a Box as PG canonical (ux,uy),(lx,ly). PG normalises the corner order on input; we trust the engine’s constructor has already normalised so the field order here is the canonical upper-right + lower-left.

format_pg_lsn

Render an LSN in PG’s %X/%X form (uppercase hex, no zero-padding).

format_point

v7.37.5 ε — render a Point as PG canonical (x,y).

format_polygon

v7.37.5 ε — render a Polygon as PG canonical ((x,y),...).

format_range_text

v7.17.0 Phase 3.P0-38 — render a Range value as its canonical PG text form. Re-exported via format_range_text for use from spg-server’s pgwire layer.

format_text_2d_text_pub

pg_guc_boot_value

v7.39 (tz epic) — host-injected IANA timezone lookups (the no_std engine can’t read the system zoneinfo directory; spg-tzif is the std-side implementation). All instants are MICROSECONDS. UTC offset (µs east) of a zone at a UTC instant; None = unknown zone. v7.39 (round 534) — the compiled-in default PG18 reports for a configuration parameter, for the wire’s own SHOW shortcut.

silent_for_update_count

v7.37.14 (A2.5-stub) — read the process-wide silent-FOR-UPDATE counter. Engines / spgctl / monitoring use this to surface “how many advisory row locks did the workload ask for since process start”. Returns 0 if no FOR UPDATE / FOR SHARE clause has hit the parser yet.

substitute_placeholders

v7.16.0 — walks a parsed statement and replaces every Expr::Placeholder(N) with the corresponding params[N-1] re-encoded as an Expr::Literal. Used internally by Engine::execute_prepared AND surfaced for the spg-embedded WAL path (which needs the bind-final AST so replay sees a simple-query-shaped statement, not a $1-shaped one). Errors when a placeholder references an index past the params slice.

Type Aliases

ActivityProvider

v6.5.2 — provider callback type. Fresh snapshot returned each call; engine doesn’t cache the slice.

AuditChainProvider

v6.5.3 — chain-table provider + verifier. spg-server registers fn pointers that snapshot / verify the audit log. verify returns (verified_count, broken_at_seq) — broken_at_seq is -1 on a clean chain.

AuditVerifier

BackendCountFn

v7.39 (pg_stat knife A) — host-provided live connection count for pg_stat_database.numbackends.

BackendPidFn

v7.39 (read01 pgstatfuncs.c) — host-provided identity of the CALLING connection for pg_backend_pid() / the pg_stat_activity self-join. The host reads a connection-thread-local set at session start; the no_std engine just calls through. None (embedded) → pid 1.

BackendSignalFn

v7.39 (round 318, V51) — host-provided connection control. terminate false = cancel the target’s running statement (PG pg_cancel_backend, MySQL KILL QUERY); true = also close the connection (PG pg_terminate_backend, MySQL KILL CONNECTION). Returns whether a connection with that id exists — the engine has no registry of its own, so the answer has to come from the host that accepted the sockets. None (embedded, no connections) ⇒ nothing to signal.

ClockFn

The execution engine. Holds the catalog and (later) other server-scope state. Engine::new() is intentionally cheap so callers can construct one per database, per test. Function pointer that returns “now” as microseconds since Unix epoch. The engine is no_std, so it can’t reach for std::time itself — callers (spg-server, the sqllogictest runner) inject a concrete implementation. None means NOW() / CURRENT_* raise Unsupported.

MonotonicNowFn

v7.17.0 Phase 2.3 — monotonic time source for deadline-aware cancellation (PG statement_timeout). Returns microseconds since some host-stable monotonic origin (typically the first call into Instant::now() on the server). The engine never calls Instant::now() directly so the crate stays #![no_std].

SaltFn

Function pointer that produces 16 cryptographically random bytes. Like ClockFn, the engine is no_std and can’t reach for /dev/urandom itself — host (spg-server) injects an OS-backed source. None means SQL-driven CREATE USER falls back to a deterministic salt derived from the username (acceptable in tests; the server always installs a real RNG so production paths never see this).

SlowQueryLogger

v6.5.6 — callback signature for slow-query log emission. Called with (sql, elapsed_us) once per successful execute that crosses the threshold.

TzAbbrevFn

Zone designation (“JST”, “EDT”) at a UTC instant.

TzAllFn

v7.39 (round 502) — every zone the host knows at a UTC instant, as (name, abbrev, utc_offset_secs, is_dst).

TzCanonFn

Canonical zone spelling (“asia/tokyo” -> “Asia/Tokyo”).

TzLocalizeFn

Local wall-clock µs -> UTC µs with PG’s DST disambiguation.

TzOffsetFn

WalLsnFn

v7.39 (round 476) — the WAL’s current byte position, as a PG LSN.