turnloop-postgres 0.1.0-alpha.7

Pull-driven sans-I/O PostgreSQL client protocol
Documentation

turnloop-postgres

A pull-driven PostgreSQL v3 client core, built on postgres-protocol frontend messages, framing and SCRAM. It never opens a socket, starts a thread, reads a clock, schedules a timer or generates entropy. The sans-I/O code forbids unsafe.

Getting started on turnloop

Enable turnloop-postgres = { version = "0.1.0-alpha.2", features = ["turnloop"] } and use the asynchronous module: Client::connect, query, execute, copy_in/copy_out and notification. The default feature set remains sans-I/O. The adapter reuses turnloop-io; the host owns LocalExecutor and calls turn. Spawn local futures through its handle and keep their JoinHandles until completion.

cargo run -p turnloop-postgres --features turnloop --example turnloop

The complete example connects to 127.0.0.1:5432 by default; TURNLOOP_DB_ADDR overrides that development endpoint. All operations take an absolute turnloop_io::Instant deadline, shared across authentication, I/O and retries. Callbacks borrow row/reply storage; copy values only when retaining them. Dropping a pending operation closes its transport before a pool can reuse it.

For TLS, set the protocol's TLS mode and provide turnloop_tls::asynchronous::ClientTls with the trust configuration, verified server name and current Unix seconds. Certificates are verified; a requested TLS upgrade without a configuration fails. Native TCP and WASI 0.2 sockets share the same driver. Browser raw TCP is unavailable; Windows accepts a host-provided Backend until the repository's production IOCP provider is integrated.

Pool::new accepts the existing pool::Config (max, max_idle, idle/acquire limits, max_uses); acquire(deadline) returns an exclusive lease. A transaction still open at release destroys the connection. Named ExtendedQuery values use the core prepared-statement cache. notification(deadline) is the async next operation after LISTEN; query callbacks also receive interleaved notifications. copy_in lends a chunk writer with finish; dropping it closes the session. copy_out lends each chunk to a callback without buffering the complete transfer. cancel_token().cancel(...) sends CancelRequest over a separate connection. TLS authentication prefers SCRAM-SHA-256-PLUS. The client derives RFC 5929 tls-server-end-point data from the verified peer leaf; an explicit ConnectOptions::channel_binding overrides that digest. RSA/ECDSA SHA-256/384/512 and RSA-PSS are supported; MD5/SHA-1 signatures use SHA-256. Unsupported algorithms (such as Ed25519) or absent peer certificates fall back to SCRAM-SHA-256 with the n,, GS2 header. Config::channel_binding_required rejects that fallback.

Driving a connection

  1. Construct Connection::new(Config { .. }). The host resolves the address, connects TCP/Unix transport and supplies any absolute connection deadline. Construction already queues output: output() holds the StartupMessage (or the SSLRequest when ssl is Prefer/Require) before any transport exists. Do not assume it starts empty; step 2 must run first once connected.
  2. Transmit output(), starting with those bytes. Acknowledge only actually written bytes with consume_output(n). Retain the borrow until write completion, or copy into a reusable host write buffer; never hold a raw pointer while mutating the core.
  3. Feed plaintext with receive(bytes), then repeatedly pull next_event() until it returns None. Flush any newly generated output before reading again.
  4. UpgradeTls is a hard boundary. Finish the existing plaintext write, perform and verify TLS in the host. Derive binding with turnloop_tls::tls_server_end_point from the verified peer leaf, then call tls_established_with_channel_binding(binding.is_some()). The original tls_established() acknowledges TLS without binding data. TLS records never go into receive. Prefer mode can fall back after N; Require cannot.
  5. For ScramNeeded, construct the reexported upstream ScramSha256 in the host (its constructor reads entropy) from connection.config().password, then call start_scram. The host needs no second copy of the password. Until it does, next_event() returns Error::State rather than Ok(None), so an ignored ScramNeeded fails visibly instead of stalling. PLUS needs ChannelBinding::tls_server_end_point containing the certificate digest defined in RFC 5929. For plain SCRAM use ChannelBinding::unsupported(). The core checks the mechanism/binding selection and verifies the server signature. Iterations are capped by configuration.
  6. Schedule next_timeout() in the host. Call handle_timeout(now) with supplied monotonic time. No method obtains time implicitly.
  7. On transport EOF, TLS failure, or an error from receive/next_event, call abort(error) and drain events. For a transport failure pass Error::transport(&io_error) (or Error::Transport(Some(TransportFailure::new(kind, code, message)))): the host's kind, error code and text (for example connect ECONNREFUSED) reach every Outcome::Aborted and the Closed reason. Error::Transport(None) keeps pg's generic message. Do not resume parsing after a protocol error. Each accepted token yields one terminal Completed; Error and command tags are informational and must not separately settle a promise. Aborting drains all tokens, then emits one Closed. Rejected command calls accept no token.

Every event borrows storage until the next mutable call. Materialize JS rows, errors, notifications and fields before advancing, or retain them with Event::into_owned(): the resulting OwnedEvent holds OwnedRow, OwnedFields and OwnedServerError (also available from Row/Fields/ServerError::into_owned), each one copy of its wire bytes. OwnedEvent::as_event() lends it back as the borrowed Event, so one handler serves both forms. ParameterStatus events allow hosts to retain whatever session parameters they need; the core does not copy an unbounded parameter dictionary. No event calls a host callback itself.

Commands and ordering

query(token, sql, deadline) supports multiple simple-protocol statements; CommandComplete separates results and includes the server's actual row count. execute(token, ExtendedQuery { name, sql, oids, params, result_formats }, deadline) writes Parse/Bind/Describe/Execute/Sync. A nonempty name is cached after ParseComplete. Reusing a name with different SQL/OIDs is rejected. Reuse before ParseComplete returns backpressure. Failed parses can be retried. Each extended operation has its own Sync, so one server error does not discard later tokens. There is no portal cursor API or automatic statement eviction yet.

CopyIn requests host-provided chunks via copy_data then copy_finish, which can send CopyFail. COPY OUT emits CopyOut, CopyData, CopyDone, a command tag and the query completion. Submit COPY when no earlier or later pipelined command can interfere; interactive COPY and queued query bytes cannot share the input phase. CancelRequest bytes go on a new transport, using the saved backend PID/secret; they never go on the active query stream. end requires drained queries, sends Terminate, and reports Closed after output acknowledgement.

LISTEN/NOTIFY uses ordinary SQL commands and independent Notification events. Transactions use SQL; ReadyForQuery tracks Idle/InTransaction/Failed accurately.

pg conversion contract

types::decode(oid, format, bytes) decodes common scalar and array wire types. The core has no JavaScript objects; the adapter applies these policies.

PostgreSQL type Core representation pg default JS representation
int2/int4 Int(i32) Number
int8 exact Int8(i64) decimal string
float4/float8 Float(f64) Number, including NaN/infinity
numeric exact Numeric string, including binary scale string
bool Bool boolean
text/varchar/name/char borrowed Text string
bytea borrowed binary or decoded text bytes Buffer
date/timestamp/timestamptz ISO text or PG epoch days/microseconds Date (local-time policy for date/timestamp; instant for timestamptz)
json/jsonb JSON text (jsonb version removed) host JSON.parse
uuid UUID text (binary formatted canonically) string
arrays of these nested Array; NULL preserved nested JS arrays
unknown OID Unknown { oid, text } (never Text) or tagged binary Raw custom parser or string fallback

JS Date milliseconds discard microseconds; the host must handle timezone, BC dates and infinity. Binary temporal values retain PostgreSQL's 2000 epoch and infinity sentinels. Lower array bounds are discarded like pg. The generic codec uses the same exact numeric semantics in both formats; pg-types' historical binary numeric/array parsers have inconsistencies, so byte-for-byte JS binary parser parity is not claimed. Host custom parsers receive OID, format and bytes.

ServerError exposes every field via fields/get, plus named accessors. Map internal_position→internalPosition, internal_query→internalQuery, context→where, data_type→dataType. Position and line remain strings as in pg. Metadata includes real dataTypeSize/dataTypeModifier, unlike the old sqlx binding's -1 placeholders. Callback/promise scheduling, rows-as-objects, rowMode arrays, URL/environment defaults, host/port and socket options stay in Perry. Existing Perry int8/numeric conversions differ from pg and are not copied.

Pool and allocation

pool::Pool is lazy, bounded, FIFO for waiters and LIFO for idle reuse. The host executes Connect/Close events and reports connected/connect_failed/closed. Checkout/checkin uses generation-checked IDs and token-bearing leases. Idle expiry, min, max, maxUses, lifetime, explicit destruction, checkout deadlines and drain-on-end are implemented. Host dispatch supplies nextTick, event emitter behavior and allowExitOnIdle/ref-unref. This is not a JS Pool wrapper.

RX/TX buffers and pending queues retain capacity. Borrowed rows and metadata allocate nothing; named registration, SCRAM/startup, pool growth and owned result materialization can allocate. Bytea text decoding, binary UUID/numeric, arrays and into_owned allocate their result representation. Allocation tests measure 1,000 warmed queries with rows and 1,000 cached executions, each with zero allocations. Buffers may grow for larger workloads; no constant-size claim.

Tests

cargo test -p turnloop-postgres runs protocol, pool, conversion and allocation tests. Real-server tests are explicitly ignored by default; missing fixture variables panic when those tests are selected. Run from the workspace:

python3 scripts/test-servers.py run cargo test -p turnloop-postgres --test server -- --ignored

The script creates private data/certificates/logs under .tools, uses random loopback ports, and stops its own servers in a finally block. start/stop are also available. See LANE_REPORT.md for actual results and server blockers.

Browser builds export a host-supplied Instant::from_duration(ticks); native and WASI builds reexport std::time::Instant. Use one monotonic epoch per connection and pool. Map the browser core loop’s Instant through as_duration/from_duration. The protocol never calls performance.now or std Instant::now. Browser runtime execution is not yet verified; only target compilation has run.

Getting started on turnloop

Enable the turnloop feature and construct asynchronous::AsyncConnection::new with a connected turnloop::AsyncIo (TCP/pipe) or turnloop_tls::TlsStream and this crate's sans-I/O Connection. Submit commands through core_mut(), then next(&executor_handle, |event| { /* consume the event */ Ok(()) }).await. Callbacks finish consuming borrowed rows before the next event. Authentication, entropy and TLS-upgrade requests are explicit events; acknowledge them through the core. For an upgrade, into_parts preserves the stream, core and unread bytes. Apply turnloop_io::deadline using the core's next deadline. Dropping a pending drive future closes the stream and aborts the core; terminal core events remain available for draining. See turnloop-io for the shared adapter ownership pattern.

Terminal server failures

ErrorResponse with nonlocalized severity FATAL/PANIC (or S when V is absent) terminates the session immediately. Connection::next_event() returns Error::ConnectionAborted(ConnectionFailure); its server_error() view retains SQLSTATE, message and all other fields. Close the transport and drain the core's one Outcome::Aborted per pending token, followed by Closed. EOF/abort does not replace the original diagnostic. Startup rejection is terminal too.

The async client closes its stream and returns io::ErrorKind::ConnectionAborted with the typed turnloop_postgres::Error as its inner error. Such a client is not reusable. Ordinary statement ERROR stays borrowed and becomes Outcome::ServerError only after ReadyForQuery; the session can then continue.

Error and Outcome are Clone, no longer Copy. Terminal diagnostics share one owned copy of the wire fields; cloning pipeline aborts allocates nothing. Successful queries and reusable statement errors retain zero steady allocations.