turnloop-mongodb
Host-driven MongoDB protocol components. The default sans-I/O library opens no sockets, reads no clock, starts no threads and uses no async runtime. It forbids unsafe code. BSON values stay BSON values; converting them to Perry's JSON strings or JS result shapes is the host's job.
The crate currently provides connection/command/operation state machines, topology and pool components, and explicit session/transaction/change-stream helpers. It is not a drop-in implementation of every Node MongoClient feature. See the root LANE_REPORT.md for the precise coverage and remaining conformance work.
Getting started on turnloop
Enable turnloop-mongodb = { version = "0.1.0-alpha.2", features = ["turnloop"] }
and use the asynchronous module: Client::connect/command/cursor and Cursor::next/close. The default feature set remains sans-I/O.
The adapter reuses turnloop-io; the host owns LocalExecutor and calls turn.
Spawn local futures through its handle and keep their JoinHandles until completion.
The complete example connects to 127.0.0.1:27017 by default;
TURNLOOP_DB_ADDR overrides that development endpoint. All operations take an
absolute turnloop_io::Instant deadline, shared across authentication, I/O and retries.
Callbacks borrow row/reply storage; copy values only when retaining them.
Dropping a pending operation closes its transport before a pool can reuse it.
For TLS, set the protocol's TLS mode and provide
turnloop_tls::asynchronous::ClientTls with the trust configuration, verified
server name and current Unix seconds. Certificates are verified; a requested TLS
upgrade without a configuration fails. Native TCP and WASI 0.2 sockets share the
same driver. Browser raw TCP is unavailable; Windows accepts a host-provided
Backend until the repository's production IOCP provider is integrated.
The client schedules SDAM heartbeats with absolute turnloop timers, owns CMAP
pools, and uses the existing Operation coordinator for retryable reads/writes.
last_retry_count reports actual retries of the last command. cursor returns
an async next(deadline) stream with server affinity and backpressure. close
awaits killCursors; drop queues bounded-deadline cleanup while the client remains
alive. Call close when acknowledgement is required. Native mongodb+srv looks
up SRV/TXT through turnloop's blocking pool and validates domains/TXT options in
the core. WASI 0.2 supports A/AAAA through ip-name-lookup; its standard interface
has no SRV/TXT capability, so use explicit resolved seed URIs there.
Host entropy
The sans-I/O core reads no OS entropy and no clock, so every random or unique value
comes from the host. Two are easy to miss because nothing fails locally: a weak SCRAM
nonce authenticates anyway, and an insert without _id succeeds against a server.
| Obligation | Where it goes | What to supply |
|---|---|---|
| SCRAM client nonce | Connection::connected(now, nonce) |
For a connection with credentials, a fresh CSPRNG nonce per connection: at least 16 printable ASCII bytes and no , (base64 of 18 random bytes works). Scram::new rejects a short or unprintable nonce; it cannot detect a reused or predictable one. Pass "" without credentials. |
Document _id |
each inserted document, before encoding | Every document sent by insert (or through BulkBatcher) must already carry _id. The server otherwise creates one the host never learns, where the Node driver would have generated it locally and returned it as the inserted id. Create one ObjectIdGenerator::new(random, counter) per process with 5 random bytes and a random 24-bit counter start, then call generate(unix_seconds) for each document. |
| Session identity | Session::new(id), RetrySession { id, .. } |
A random RFC 4122 UUID per server session. SessionPool::checkout returns None when a new one is needed. |
| Server selection | Topology::choose / select_reusing |
Two random u64s per selection, for an unbiased pick within the latency window. |
The turnloop feature's asynchronous client fills the nonce, session identity and
selection entropy from rustls' ring SecureRandom. Adding _id remains the caller's job
in every mode.
Driving a connection
- Parse
Options. Formongodb+srv, execute bothresolution_requests()and provide the answers toresolve(). The host opens the resulting address using its transport. - Construct
Connection, callconnected(now, nonce). A nonce must be supplied by a host CSPRNG for authenticated connections (see Host entropy).UpgradeTlsmeans finish certificate-validated TLS on the transport and calltls_established()before sending Mongo bytes. - Write
transmit()and acknowledge only bytes actually written withconsume_transmit(n). The slice must not be retained across mutating calls; an adapter using completion I/O keeps the connection immobile until write completion. - Feed plaintext bytes through
receive(). It returns the consumed prefix: loop on the remainder. It deliberately stops at a frame boundary. Drain events and release a complete reply before feeding another frame. Partial reads and writes are normal.accepts_receive()is the same checkreceive()makes, so a host can ask whether to read instead of tracking anexpecting_replyflag of its own. - On
Ready, submit one command. An accepted token is settled exactly once: byFailed, byUnacknowledged, or byrelease_reply()after itsReplyevent; rejected submissions receive no completion.Replyis a borrowed view obtained withreply(). Consume or copy it, thenrelease_reply(). - Pass transport errors to
fail(). Drivehandle_timeout(now)atnext_timeout(). A timed-out connection must be physically closed by the adapter.fail()settles an outstanding command, or a reply not yet released, withFailed: the reply is revoked and a still-queuedReplyevent for it is withdrawn.close()cancels an outstanding command once, followed byClosed, but a reply already received remains readable after close until released.
Instant is std::time::Instant on native/WASI. Browser Wasm uses HostInstant, created
from host monotonic ticks with Instant::from_duration. It has no now() method. The
browser host must supply a permitted transport (e.g. a tunnel); browsers do not expose
raw MongoDB TCP sockets.
Commands and results
Command and WriteModel retain BSON buffer capacity. Options are wire fields, such
as projection on find, fields on findAndModify, and new for return-after. Inputs
are borrowed RawDocuments. Callers must not reuse a builder while its raw view is in use.
use ;
let filter = try_from.unwrap;
let options = try_from.unwrap;
let mut command = new;
command.find.unwrap;
assert_eq!;
Insert/update/delete use OP_MSG document sequences named documents/updates/deletes.
BulkBatcher splits borrowed models by negotiated count, BSON and wire size limits;
pass the final decorated body when calculating overhead. A failed write can arrive
inside ok: 1: a duplicate key is reported in writeErrors and an unmet write concern
in writeConcernError. WriteResult::parse is the verdict — it runs
Error::from_response first and fails on either, so no separate check is needed.
WriteResult::decode keeps both as data; check succeeded() on its result. BulkResult
uses decode and preserves original error indices across batches. It exposes aggregate wire counts; the caller
maps them and inserted/upserted IDs to the desired JS result structure. Missing _id
values must be added before sending with ObjectIdGenerator; see
Host entropy.
CursorBatch::rows() yields borrowed RawDocuments without per-row allocation. Cursor
tracks namespace, server affinity and client limit. needs_kill() means issue
killCursors before discarding its live id. The host retains cursor affinity across
getMore calls and never retries getMore as a retryable read.
Operations, topology and pooling
Operation retains an OP_MSG template. Its actions are Select, Checkout, Send, Waiting,
Complete and Failed. Use Topology::candidates_deprioritized and choose for selection,
Pool for checkout, and Operation::send for dispatch. On replies call
Operation::response; on failures call failed. Only terminal outcomes settle the
user's operation. An initial retryable failure produces a new Select action; the
second attempt keeps lsid/txnNumber and gets a fresh connection request id.
A retryable write needs a host-supplied RetrySession with a unique UUID and a txnNumber
incremented once per logical write. Reads can also use this session input. Raw explicit
session fields are preserved and validated against supplied identity. Multi-update,
multi-delete, transaction commands, getMore, $out/$merge and generic runCommand are
excluded from automatic retry. Operation timeouts/cancellation require the adapter to
close an outstanding connection before reusing its transport or pool lease.
Topology returns Check, ServerAdded/Removed/Changed, TopologyChanged and ClearPool
requests. Check each server using its own monitor connection; report hello responses
and measured RTT via update, and application errors with their connection generation
via application_error. Pool clearing must follow those requests. Host callbacks for
connections removed by a pool clear are stale and must not be made available again.
Call operation_started/finished to maintain the power-of-two selector's operation counts.
Pool starts paused; mark it ready after successful monitoring. Execute Connect/Close
requests, and report a connection ready only after its TLS/handshake/authentication
finishes. Checkout tokens are FIFO. Checked-out connections from old generations close
on checkin. Poll and handle its next_timeout for waiters and idle expiry.
Options::raw retains accepted write/read concern URI settings. Use Command::apply_client_options for URI defaults; existing command values take
precedence. The adapter handles database/collection inheritance. Connection itself
does not add concerns or read preference to arbitrary user commands.
Sessions and change streams
Session decorates explicit sessions and transactions, tracks transaction numbers,
gossips cluster time, provides causal-read decoration, caches recovery tokens, and
builds commit/abort. SessionPool reuses clean identities until their host-driven expiry.
TransactionEnd coordinates one commit/abort retry and final error labeling; failed
abort results are suppressed after that attempt as the specification requires. Retried
commit uses majority, preserves configured wtimeout, and supplies 10 seconds if absent.
Mongos pin/unpin is explicit. Snapshot sessions, a withTransaction callback/replay
controller and complete transaction-spec conformance remain work.
ChangeStream tracks resume tokens, startAfter/resumeAfter transition and one-resume
error policy for getMore failures. Initial aggregate errors are terminal. Call observe_document for each emitted event and finish_batch only after
the whole batch is consumed. Build the new aggregate with stage() on resume; preserve
user pipeline/options in the adapter. Closing cursors and driving getMore remain
explicit. Full stream lifecycle orchestration is not yet included.
Verification
From the workspace root: scripts/verify-mongodb.sh. Real servers:
python3 scripts/test-servers.py run. The latter starts five isolated MongoDB processes
(standalone, three replica members, TLS standalone), runs the blocking std socket /
rustls tests, then stops its children and verifies all private ports closed. Replica-set
initiation, users and all administrative commands use this crate; no Mongo shell or
official Mongo driver is used. start / stop are also available for debugging.
Official unmodified JSON fixtures are in tests/spec. COMMIT pins their upstream revision and LICENSE.md retains their separate upstream license. Suites cover SDAM transitions and application errors, server selection and max staleness. They exclude load-balancer, unified integration, event-monitoring and logging fixtures; they are not the complete MongoDB driver specification suite.
Getting started on turnloop
Enable the turnloop feature and construct asynchronous::AsyncConnection::new
with a connected turnloop::AsyncIo (TCP/pipe) or turnloop_tls::TlsStream and
this crate's sans-I/O Connection. Submit commands through core_mut(), then
next(&executor_handle, |event| { /* consume the event */ Ok(()) }).await.
Callbacks finish consuming borrowed rows before the next event. Authentication,
entropy and TLS-upgrade requests are explicit events; acknowledge them through
the core. For an upgrade, into_parts preserves the stream, core and unread bytes.
Apply turnloop_io::deadline using the core's next deadline. Dropping a pending
drive future closes the stream and aborts the core; terminal core events remain
available for draining. See turnloop-io for the shared adapter ownership pattern.