Crate happenstance_cloudflare
Expand description
Cloudflare Durable Object adapter for happenstance — the workspace’s !Send
instrument.
§Status: bound, implemented, packaged — and released at 0.2.0
This crate depends on worker and talks to a real Durable Object’s
SqlStorage. js and sql_storage are bindings rather than models:
exec is worker::SqlStorage::exec, a cursor is a SqlStorageCursor, and
a thrown value is a worker::Error kept live behind an Rc.
Every EventStore body is real —
migrate, append, head, contains_event_id and read all execute SQL
against the object’s own storage — and no unimplemented body remains
anywhere in the crate. The scoped allow of the clippy::todo lint that
used to stand below the module list left with the last of them, which is
what it was written to do; the lint is denied workspace-wide, so the gate
now fails on the first one that comes back.
There is now something to hang a store off, too. [host] stands up one
Durable Object’s state — real SQLite through Node’s node:sqlite, reached
by the same State::storage().sql() a #[durable_object] class calls — and
it is pub so that a conformance target, which is a second compilation
unit, can name it. It is a test-and-example surface and not a second
public API: it carries #[doc(hidden)], the adapter stays a library type any
real Durable Object class can hold, and [host]’s own documentation says
what that visibility does and does not promise. Every item in it panics on a
real Workers isolate — process.getBuiltinModule('node:sqlite') does not
exist there — so whether it travels to a consumer at all was left as
publish-ready-crate’s decision, and hiding it kept that a decision about
removing an undocumented item rather than about breaking a published one.
That decision is taken, and it is that the module travels, hidden. The
alternative was to strip it before the crate became publishable, and it
fails on reachability rather than on taste: tests/ is a second compilation
unit that links this crate’s public surface and nothing else, so the
conformance target and the fixture contract can only name the host if it is
pub here. Removing it would take the adapter’s only executed conformance
run with it. What #[doc(hidden)] then buys is that no consumer is told
about an item that panics where they will run it, and that deleting it later
is the removal of an undocumented item rather than a breaking change. The
third option — a test-host Cargo feature — buys the same thing and costs a
cargo hack powerset dimension on every run of the gate, so it stays
available and unspent.
§Status: what publication this crate is ready for, and what it is not
Ready to be packaged, and packaged: both licence texts and a README sit
beside the manifest inside the package directory, readme = "README.md" is
stated rather than left to auto-discovery, publish = false is gone, and the
crate’s name is in PUBLISHABLE (xtask/src/package.rs) — which is the half
that matters, because it puts this crate inside the gate step that asserts
all three files are in the artifact on every run, forever, rather than
leaving it a fact about one working tree. reconcile fails in both
directions, so the manifest flag and the list cannot drift apart.
Released at 0.2.0, and that changes what this page owes a reader. Two
commitments arrive with the version and neither is optional now: the
EventStore clauses marked [FROZEN] in the specification are
semver-binding here as everywhere, and this crate carries the workspace’s
MSRV of 1.97.1, a promise from this release (ADR-0037), bounded by ADR-0067.
ProjectionStore was not frozen at that release and is since ADR-0063; this
crate does not implement it at all, so neither state touched it.
Until this release the crates.io name was held by the 0.0.0 placeholder
cargo xtask reserve generates — a standalone crate sharing nothing with this
workspace but its metadata — for the reason xtask/src/reserve.rs gives:
publishing the real crate at a real version would have made this API
semver-binding before the phases that freeze it did so deliberately, against
evidence. Those phases have run. The placeholder has done its job and the real
crate is what 0.2.0 puts on the registry.
§Conformance: what has run, where, and what is deliberately not asked to
The event-store family runs in full, on this target, inside the gate.
crates/happenstance-cloudflare/tests/durable_object_conformance.rs is three
lines — happenstance_testkit::event_store_conformance! with
emit = happenstance_testkit::emit_wasm and a CloudflareFixture — and
cargo xtask ci executes it on wasm32-unknown-unknown under
wasm-bindgen-test-runner. The rule set is not a claim this file makes: the
macro expands for_each_event_store_rule!, which is the one place the list
is written, and xtask’s registry asserts every name that enumeration
declares out of the target’s own --list before the run starts. A
wasm32-only subset is not a thing anybody here has to be trusted about.
§What it runs against, stated precisely
A DurableObjectState-shaped shim shipped in this crate, backed by Node’s
own node:sqlite — [host], and its documentation is the long form. The
adapter above it is unmodified: worker::State::from(…) →
state.storage().sql() → worker::SqlStorage::exec, with worker’s real
wasm-bindgen externs in the middle and real SQLite underneath. What is
doubled is the runtime, never the adapter.
It is not workerd. No wrangler, no miniflare, no
vitest-pool-workers, no isolate, no eviction, no hibernation and none of
the platform’s own storage ceilings. Getting a workerd-class runner inside
cargo xtask ci is an escalated blocking finding this project raised rather
than absorbed, and it is ADR-0023’s to settle. Two things therefore remain
provisional and are marked as such where they are stated: the three capacity
limits below, and what a real isolate restart would do to an acknowledged
write. Everything else — every rule of the enumeration, against a !Send
store, on wasm32-unknown-unknown, in the same terminal scroll as the rest
of the gate — genuinely executed.
The fixture lives in tests/support/mod.rs rather than in src/, because
happenstance-testkit is a dev-dependency and an impl Fixture here would
put the suite that measures this adapter into the runtime graph of every
consumer of it. Its own contract — one instance is one object, two instances
share nothing, connect() is a second handle onto that object — is held
honest by tests/fixture_contract.rs.
§The concurrency family is not invoked, and that is a reason rather than a
§silence
event_store_concurrency_conformance! binds F::Store: EventStore + Send
and its module is #[cfg(not(target_arch = "wasm32"))], because the family
needs threads to spawn. This adapter’s store is !Send by construction — it
is the reason the bare flavour exists — and wasm32-unknown-unknown has no
threads, so a !Send adapter cannot invoke that family and is not
expected to.
Read that as a narrowing of the word conformant and nothing more. It costs
this adapter nothing it could otherwise have had: a Durable Object is a
single-threaded actor with exclusive ownership of its storage, so there is no
second writer for a race to elect a winner between, and the interleaving the
family exists to stress is not reachable here at all. What the event-store
family does run against this store includes
interleaved_appends_on_one_handle_elect_one_winner and
a_live_read_stream_does_not_block_an_append, which are the single-threaded
shapes of the same question.
§The capacity limits this store declares
VT-21 asks a store to document its actual limit. What follows is this
adapter’s declared refusal policy, not a measured physical wall — the
paragraph under the table says exactly what was and was not observed, and
it is the first thing to read here rather than the last. The numbers are
enforced by event_store::CloudflareEventStore’s own ceiling check
before any SQL is issued, which is what lets a refusal name which
ceiling was crossed — a refusal classified after the fact from a thrown
storage error could not, and a refusal arriving after some rows had landed
would be a partial batch.
| Limit | Value | Refused as |
|---|---|---|
payload (data) | 1,048,576 bytes (1 MiB) | AppendError::ExceedsStoreLimit { limit: StoreLimit::EventDataLen, .. } |
| tags per event | 1,024 | … limit: StoreLimit::TagsPerEvent |
| events per append | 1,024 | … limit: StoreLimit::EventsPerBatch |
metadata per event | 524,288 bytes (512 KiB) | … limit: StoreLimit::MetadataLen |
| tags per call, summed over its events | 262,144 | … limit: StoreLimit::TagsPerBatch |
The first three clear their guaranteed minimum by a wide margin — sixteen,
sixteen and eight times — the metadata ceiling has none to clear
(ADR-0043), and the tags-per-call ceiling clears its own 32 times. The first
four are each a stated ceiling, not the physical maximum: no
physical wall was observable on the executing host at 8 MiB of payload,
16,384 tags or 8,192 consecutive inserts, and that host is a Node process
rather than workerd, so it does not enforce the platform’s documented caps.
They are this adapter’s own refusal policy, carved from Cloudflare’s
documented row limit of 2 MB, read as 2,000,000 bytes; one row at all four
ceilings is 1,835,264 bytes and fits (ADR-0093). The experiment’s
README records the failure to locate the wall as its central finding.
The fifth is placed against a wall a deployed object hit: one call of
512 events of 1,024 tags ran past the object’s storage timeout and reset
it (#371), and 262,144 is half that (ADR-0100). ingest
counts every foreign event and every compensation toward it.
What was measured, on this host and reproducibly: the declared value is
accepted and read back byte-for-byte, and one more is refused as
ExceedsStoreLimit naming the ceiling it crossed. That is exactly what CF-40
asks a store to guarantee — it does not ask for the largest value the store
could ever take, and an unstable exact maximum is how a green run becomes a
flaky one. Locating the real wall needs a workerd-class runner, which is
the escalated blocking finding ADR-0023 owns. The derivations and the two
consecutive identical runs behind them are in
experiments/durable-object-limits/README.md.
§The query widths this store does not refuse
A separate table on purpose, because it answers a different question. Every
row above is a value the store refuses, and names the StoreLimit it
refuses with. What follows is the opposite: a query wider than either number
is chunked and merged, never refused, because a Query bounds nothing by
design — VT-23 requires every store to evaluate at least 128 items and puts
no ceiling above that — and a query-item refusal is not an append outcome, so
the contract has no variant to report one through.
| Width | Value | What it is |
|---|---|---|
MAX_QUERY_ARMS_PER_STATEMENT | 5 | index arms in one statement: workerd’s compound-SELECT wall of 5 terms, exactly |
MAX_QUERY_PARAMETERS_PER_STATEMENT | 90 | bound parameters in one statement’s arms: workerd’s wall of 100, less the read wrapper’s 5 and headroom |
They are published because VT-23 does not ask for them. Unlike VT-21,
VT-22 and VT-24, VT-23 imposes no documentation obligation — a store may
refuse a wider query and need not say where. That silence is exactly how two
adapters shipped under one contract at one version can have materially
different query capability with nothing on either page to compare. These
numbers are not happenstance-sqlite’s 400 and 30,000, and need not be:
they are workerd’s walls, measured inside it (ADR-0079), where the sibling’s
are SQLite’s compiled defaults. An item binds 0 to 3 parameters whatever its
width, so a 128-item query is 26 statements here and one there; both answer it.
planned_statement_count
is how a caller asks the question directly, before deploying rather than
after: it is the same call both paths plan with, counted.
What does differ is the merge, and it differs for this runtime’s reason.
A Durable Object is a single isolate with a real memory ceiling, so the read
path here sorts and truncates after every statement of a page rather than
after all of them — one page plus one chunk resident, however wide the query
— which is exact rather than approximate. src/query_sql.rs carries the
argument.
§The model family is not in the graph on this target
It sits behind the testkit’s off-by-default proptest feature, and both
fixtures::strategies and model carry a target condition on top of that
feature — because a Cargo feature is not target-scoped, so
--all-features would otherwise switch it on for wasm32, where proptest
is not a dependency at all. So it is absent here by construction rather than
by choice, and there is nothing for this adapter to opt into.
§After a point-in-time restore
A Durable Object rolled back to a bookmark rolls back this store’s events, the
row holding its StoreId and the counter that hands out positions, all
together. The restored object would then hand out EventIds it already gave
to events written after the bookmark, and a replication peer would drop the
new events as duplicates. So before the restored object accepts its first
append, the deployment calls
CloudflareEventStore::remint_identity
on every log in it.
The store cannot do this itself: the platform sends a restored object no signal, and anything the store could write to remember is rolled back with it. VT-6 permits mint-once only because this procedure is documented, which the README’s After a point-in-time restore section does in full.
§What this crate is for
It implements the bare EventStore
rather than the derived SendEventStore — happenstance-neon does too, and
for its own reason — and it is the only one whose Error is genuinely
!Send. That second conjunct is the load-bearing one, and it is what the
ES-6 verdict below actually rests on. That
makes it the sole instrument for ES-6 — “whether Error gains Send + Sync” — which the specification settles precisely because the two other
in-tree confirmations are free by construction: MemoryStoreError is
uninhabited and SqliteEventStoreError has one placeholder variant, so
neither could fail the bound if the bound were wrong.
§Findings
Made against a stand-in, and each one now either confirmed against the real API or corrected in place with the correction stated. A finding quietly deleted is a finding that will be re-discovered.
§1. A real JsValue is Send + Sync — and so is worker’s own storage
wasm-bindgen 0.2.126, src/lib.rs:168-176:
pub struct JsValue {
idx: u32,
_marker: PhantomData<*mut u8>, // not at all threadsafe
}
#[cfg(not(target_feature = "atomics"))]
unsafe impl Send for JsValue {}
#[cfg(not(target_feature = "atomics"))]
unsafe impl Sync for JsValue {}Confirmed, and it is worse than the stand-in recorded. Workers builds
wasm32-unknown-unknown without atomics, so JsValue — and therefore
worker::Error, including its Internal(JsValue) and UnknownJsError { original: JsValue, .. } variants — is Send + Sync there; and the host
build, where target_feature = "atomics" is likewise unset, gets the same
two impls. worker then writes two more of its own, on the storage handle
and on its cursor (worker-0.8.5/src/sql.rs). Four unsafe impls in the
dependency graph, all of which this crate would inherit by holding one of
those types bare. The specification’s premise for ES-6, that “an adapter
error holding a JsValue or an Rc<str> satisfies [the unbounded type] so
a spawned handler’s error cannot cross a JoinHandle”, is half wrong: the
JsValue half costs nothing, the Rc half costs everything.
That is why every JS-side value in this crate is reached through an Rc —
js::JsHandle holds Rc<JsValue>, js::JsThrow holds
Rc<worker::Error>, sql_storage::SqlStorage holds
Rc<worker::SqlStorage>. Rc<T> is !Send for every T, including a
T that carries an unsafe impl Send. The thrown value stays live; the
auto trait does not come with it. It is also not an option to mimic the
hatch: this workspace sets unsafe_code = "forbid", so an adapter
can only ever inherit that escape hatch, never write it.
§2. Stringifying a thrown value loses a capability, not information the
caller needs
js::JsThrow keeps the thrown value and can call
js::JsHandle::property; StringifiedThrow keeps String(value) and
cannot. On the one question the port makes a caller ask — was this a
conflict? — they answer identically, because a Durable Object surfaces
SQLite’s own text (UNIQUE constraint failed: event.position) through the
thrown Error’s message and exposes no numeric code. Confirmed against
the real API: worker caches name, message and code at conversion
and finds no SQLite code to cache, which is why
js::JsThrow::is_constraint_violation probes code and then falls
through to the message every time. The capability that is genuinely lost is
forward compatibility: a caller holding the live value can read a field
nobody has thought of yet.
StringifiedThrow is pub(crate) and is deliberately not linked above.
Read the paragraph as the comparison it is — the two shapes ES-6 had to
choose between — and not as an invitation, because the route it describes is
one this crate has withdrawn. The finding is unchanged by the withdrawal: it
is about what stringifying costs, which is a fact about the two shapes and
not about which of them a caller can name. Keeping the name in prose while
removing the link is the whole of the edit, and it is the point — a page that
argues for a route the API no longer offers is worse than one that never
mentioned it. js.rs carries why the type stayed and why it stopped being
public.
Now observed rather than predicted, and from the caller’s seat rather than
this module’s. The es6_reconstruction tests in this file drive a
constructed thrown value through worker’s real bindings, this crate’s real
classifier and
EventStore::append, and rebuild
the one fact a caller must branch on — conflict versus transport fault — out
of the public surface alone. They carry their own positive control, so they
are assertions that can fail rather than assertions that cannot.
§3. The conflict signal never travels in Self::Error anyway
happenstance-core lifts the DCB concurrency signal out of the adapter’s
error type and into
AppendError::ConditionViolated.
So the adapter has to classify the constraint violation before Self::Error
is constructed, whatever Self::Error is. Note the absence of a
ConditionViolated variant on
event_store::CloudflareEventStoreError — that is the finding made
structural. Stringification therefore cannot cost the caller the conflict
signal, because the conflict signal is not in the error type on any adapter.
What the error type does owe a caller is the other half of the same
branch: enough to tell a transport fault from a conflict, from a capacity
refusal, from a binding nobody wired up. A classifier that picks the right
AppendError arm and then discards the evidence satisfies every other check
in this repository and leaves that caller with nothing to act on;
the es6_reconstruction tests name that shape and reject it.
§4. The Send flavour does not imply a Send error either
send_shape::send_flavour::SendStoreWithLocalError implements
SendEventStore — Send store, Send stream, Send future — with a
!Send Error, and compiles. So ES-6’s named rule
store_error_crosses_a_join_handle is unwritable against today’s port for
every adapter, not merely for this one. See send_shape for the probe
that separates “the future is Send” from “the error is Send”.
§The ES-6 verdict
Recorded here; minted elsewhere. On the four findings above, ADR-0009’s
decision holds and this adapter is the evidence for it rather than the
exception to it: a caller recovers conflict-versus-transport from what
append hands back without Error carrying a Send + Sync bound, so the
strength belongs in a downstream marker rather than in the port. Both halves
of the clause now have an artefact in this file — the auto-trait half in the
!Send probes, the information half in es6_reconstruction — and a reader
asking what ES-6 resolved to finds both without leaving the page.
The decision atom that states it, with the alternatives that lost, is
adr-0023-and-atom-resolutions’, authored through /redkiln:kb-ingest.
Nothing under .kb/ is written by this crate, and
.kb/decisions/0009-error-send-sync.md is accepted and immutable.
§Capability limits that are not type errors
Two, and neither shows up as an error[E….]:
- A cursor is not a stable snapshot — and it did not have to be.
Cloudflare documents that a
SqlStorageCursorheld across anawait“does not provide a stable snapshot of query results”, and this crate used to record that as an unresolved choice between laziness and isolation, citing ES-9. Both halves of that were wrong, and the correction is the finding. ES-9 isfromnames a position, not an index; the clauses that carry the sample obligation are ES-11 (a read is one sample) and ES-12 (all items of one query share it), and ES-11 states outright that “laziness is therefore permitted and never required”. So there was no dilemma to resolve, only a mechanism to implement: ADR-0011’s ceiling-and-page, whichevent_store::SqlRowStreamuses. A position ceiling is captured no later than the first poll, every statement after the first is bounded by it, and each page is drained into memory before the caller can suspend — so no cursor ever spans a suspension point and the collision does not arise.sql_storage::SqlError::CursorInvalidatedremains as the report for a cursor that is outlived by another statement, which is now only reachable by a caller drivingexecdirectly. ES-11 and ES-12 name this adapter as the falsifier they were most at risk from; the ceiling is affordable here, so it does not bite, and what it costs is one extra statement per page and the rows those pages re-read. - A caught throw does not roll back the turn. A Durable Object’s
implicit transaction commits when the handler returns normally, and this
adapter converts every throw into an
Err(…)and returns normally — which is what a caller branching onAppendErrorneeds it to do, and which means the rows a failed batch had already written would commit with the rest of the turn. Isolation is not atomicity, and reading the first as the second was this crate’s own error for one milestone.SAVEPOINTis not available either: a Durable Object rejects transaction control throughsql.exec(). So all-or-none is done, explicitly, byevent_store::CloudflareEventStore’s write path discarding the positions a failed batch was assigned — exact rather than best-effort, because nothing is awaited mid-batch. When the discard itself fails, the caller is told throughevent_store::CloudflareEventStoreError::PartialBatch— and a retry is still safe after it: the rows left behind never reached theUPDATEthat stamps a batch into events, so no read,heador append guard sees them. - Positions are bounded by 2^53, not 2^64. Workers SQL widens integers
through a JS number on the way out, so a
SequencePositionaboveNumber.MAX_SAFE_INTEGERis not round-trippable even thoughNonZeroU64permits it. A stored value that crossed the line arrives back assql_storage::SqlValue::Realrather than as a narrowed integer, and is reported asevent_store::CloudflareEventStoreError::StoredPosition.
What a dropped append future does is ES-23’s question, and it is
answered in event_store’s own # Cancellation section — beside the
body it is about.
§Targets
wasm32-unknown-unknown is the target this crate exists for and the only
one where its bindings resolve to a live JavaScript heap. It also compiles
on the host. The host build is a convenience rather than evidence: nothing
outside this crate’s tests is cfg-gated, so it says only that the crate is
portable, not that a Durable Object adapter is — every worker binding it
links resolves to a stub that panics rather than to a JavaScript heap.
What the host build is good for is the one thing a contributor needs in
their inner loop: the !Send probes below run there, under an ordinary
cargo test, with no wasm toolchain at all. They run on wasm32 too — see
the twin below for why one target is not enough.
§Running this crate’s tests
Two commands, and neither alone reaches everything. The wasm32 one:
$ CARGO_TARGET_WASM32_UNKNOWN_UNKNOWN_RUNNER=wasm-bindgen-test-runner \
cargo test -p happenstance-cloudflare --target wasm32-unknown-unknown \
--lib --test durable_object_conformance --test fixture_contract \
--test wf11_memory_ceiling --test wide_query_ceiling--lib alone reaches only the cases in src/: the conformance suite is the
durable_object_conformance target under tests/. The list is the one
xtask’s WASM_TARGETS and WASM_UNIT_TARGETS rows carry for this package.
The host one is a plain cargo test -p happenstance-cloudflare: the query
planner, namespaces, !Send probes and host-only targets, never a store that
appends or reads. That is a property of the store, not a choice. Every worker
binding links on the host and resolves to a wasm-bindgen stub that panics
when called, so a SqlStorage cannot be driven off-target at all: the
tests that append and read have to be where a JavaScript heap is. What moved
with them is only what had to; the probes stayed host-reachable and gained a
target-side twin rather than being relocated.
Two things the command needs, and both fail confusingly if they are missing:
wasm-bindgen-test-runner, matchingCargo.lock’swasm-bindgenexactly — the runner refuses a mismatched schema part way through a test binary.cargo install wasm-bindgen-cli --version <locked> --locked.- Node 24. The runner’s host is Node, and
host.rs’s Durable Object shim opens Node’s ownnode:sqlite— real SQLite, the engine a Durable Object runs — withworkerd’s statement limits, an option Node 22 lacks. On an older Node every case fails at once inside the shim, which reads as “the adapter is broken” and is not.
Neither is a precondition for touching the crate: cargo test,
cargo clippy and cargo check all run on a stock stable toolchain with no
wasm target installed. The gate runs the command above for you, a target
at a time — those rows are what the wasm32 run of the conformance rules
step executes — so a regression here fails the gate
rather than waiting for someone to think of running it.
Re-exports§
pub use event_store::CloudflareEventStore;pub use event_store::CloudflareEventStoreError;pub use event_store::SqlRowStream;pub use js::JsHandle;pub use js::JsThrow;pub use sql_storage::SqlCursor;pub use sql_storage::SqlError;pub use sql_storage::SqlRow;pub use sql_storage::SqlStorage;pub use sql_storage::SqlValue;pub use event_store::CloudflareIngestStore;syncpub use happenstance_core;pub use worker;pub use happenstance_sync;sync
Modules§
- event_
store - A Durable Object-backed
EventStore, in the bare!Sendflavour. - js
- The JS boundary, bound rather than modelled: the live thrown value and the two error shapes ES-6 has to choose between.
- send_
shape - Compile probes for ES-6:
Sendon the future is notSendon the error. - sql_
storage - The Workers
SqlStorageAPI, bound — with the four shape characteristics the stand-in was built to hold, now checked against the real thing rather than asserted about a model. - sync_
http sync - The Durable Object as a sync peer over HTTP (#130):
serve_syncbehind an object’s fetch handler.
Structs§
- Table
Namespace - A validated prefix that keeps one log in its own tables.
Enums§
- Invalid
Table Namespace - Why a string is not a
TableNamespace.