Expand description
§wse-client
The Rust client of WSE: topics with
recovery, reconnects, msgpack, end-to-end encryption and signed messages,
on tokio and in the browser (WebAssembly).
It speaks the protocol of the TypeScript and Python clients to the same
servers: wse in Rust and
wse-server in Python.
[dependencies]
wse-client = "3.2"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
tokio = { version = "1", features = ["rt-multi-thread", "macros"] }§A stream of what the server sends
use wse_client::{Config, Event, Incoming};
#[tokio::main]
async fn main() -> Result<(), wse_client::Error> {
let mut config = Config::new("wss://example.com/wse");
config.token = std::env::var("WSE_TOKEN").ok();
config.topics = vec!["prices".into()];
let (client, mut events) = wse_client::connect(config).await?;
client
.send("hello", &serde_json::json!({"from": "a bot"}))
.await?;
while let Some(incoming) = events.next().await {
match incoming {
Incoming::Message(message) => {
println!("{} {:?}: {}", message.kind(), message.topic(), message.payload_value()?);
}
// Messages that will not come: reload what depends on them.
Incoming::Event(Event::Gap(gap)) => println!("{} lost {:?}", gap.topic, gap.missed()),
Incoming::Event(event) => println!("{event:?}"),
_ => {}
}
}
Ok(())
}§A handler, without copies
A Handler is called on the connection’s own task with each message read
in place from its frame; its payload decodes into the application’s types,
borrowing from the frame.
use wse_client::{Client, Config, Event, Handler, Message};
#[derive(serde::Deserialize)]
struct Tick<'a> {
symbol: &'a str,
price: f64,
}
struct Prices;
impl Handler for Prices {
fn message(&mut self, message: &Message<'_>) {
if message.kind() == "tick" {
if let Ok(tick) = message.payload::<Tick>() {
println!("{} {}", tick.symbol, tick.price);
}
}
}
fn event(&mut self, event: Event) {
if let Event::Gap(gap) = event {
println!("{} lost {:?} messages", gap.topic, gap.missed());
}
}
}
#[tokio::main]
async fn main() -> Result<(), wse_client::Error> {
let mut config = Config::new("ws://127.0.0.1:5006/wse");
config.topics = vec!["prices".into()];
let client = Client::connect(config, Prices).await?;
client.closed().await;
Ok(())
}examples/chat_client.rs talks to the chat server of the wse crate.
§What a client does
- Topics with recovery. Each topic message carries its position; the
client drops duplicates, holds what comes past a gap while it asks the
server for what is missing, and after a reconnect subscribes again from
where it left off. What cannot be recovered is reported as a
Gap. - Reconnects with backoff and jitter, a fresh token from a
TokenProvidereach time: a refused or expired token is fetched again, three times in a row at most. A fixed token that is refused, a server that refuses the session, and the application’s own close end the client. - Heartbeats both ways, and a connection silent for too long is made again.
- Wire formats. JSON in text or binary frames, compressed frames, and
msgpack (
Format::Msgpack, 24-41% fewer bytes). - End-to-end encryption with the server’s identity key pinned
(
Encryption): whatever terminates TLS in between sees nothing. - Signed messages.
Client::send_signedsigns with the client’s key for the server’s; the server’s signed messages are checked against the keys the client pins, and each carries its proof. With encryption, the client also proves its key inside the session, which binds a token that names the key to this client (Encryption::bind_session, on by default), and stops unless the server confirms the key. A signed message goes out on the connection it was signed on, or not at all. - Live messages.
Client::send_liveandtry_send_livesend on the connection that is ready now, or not at all: one the connection did not write is dropped, never sent on a later connection. For commands that must not run late (a valve, a door, an order). - The outbox. Every message sent, except a live or signed one, waits
until the server answers for it and goes again after a reconnect; a 3.2
server takes each once where it can tell, and marks it a possible repeat
where it cannot. The application hears what became of each. In memory, or in a file
that outlives the application (see below). Every message carries an
id(a UUID, or the application’s own), as from the TypeScript and Python clients. - Another user. When the token provider’s next token is another user’s, what the last user queued and did not send goes nowhere, and the topics are subscribed again from the start.
- Recovery per stream. A topic published on several servers of a cluster has a stream per server; a recovery names the client’s offset in each, and a stream that stayed quiet on the connection for longer than the server keeps one is no longer named. Time away keeps every position.
- Pause and close.
pause(reason)ends the connection, after what is queued went out, and nothing connects while any reason holds;resume(reason)for the last one connects at once and recovers what was missed.close_within(deadline)gives what is queued the deadline to go out and then stops. - Back-pressure. A handler that waits in
readyholds the server back; the stream ofconnectdoes the same while nobody reads it.
// Kept by the client. The backend names its public key in the client's
// token (`cnf`: `wse_client::public_key_jwk(&key)`).
let key = wse_client::generate_key();
let mut config = wse_client::Config::new("wss://example.com/wse");
config.token = Some("<a token whose cnf names the key>".into());
config.encryption = Some(wse_client::Encryption::pinned([
"<the server's e2e_identity_public_key>",
]));
config.signing_key = Some(key);
config.server_signing_keys = vec!["<the server's signing_public_key>".into()];
let (client, _events) = wse_client::connect(config).await?;
client
.send_signed("order", &serde_json::json!({"side": "buy", "qty": "1.5"}), Some("order-7"))
.await?;§Where every fact counts
Config::delivery with a Delivery (recover-or-resync) holds a topic that
is not continuous – from a cold start, after every reconnect, after a break
– until the application brought its state of it up to date: an
Event::Resync asks for it, and Client::resynced(round) hands on what was
held, in order. After applying a message the application calls
client.commit(&message); client.checkpoint() is what to save with its own
state and give back as Delivery::checkpoint, and recovery starts from it,
so a message can come twice: apply a repeat safely. A checkpoint of another
realm, user, consumer or schema is refused aloud (Event::CheckpointRejected),
and a message handed out before the events showed another user counts for
nobody when it is committed.
use wse_client::{Delivery, Event, Incoming};
let mut config = wse_client::Config::new("wss://example.com/wse");
config.topics = vec!["orders".into()];
let mut delivery = Delivery::new("board");
delivery.checkpoint = None; // what the application saved last
config.delivery = Some(delivery);
let (client, mut events) = wse_client::connect(config).await?;
while let Some(item) = events.next().await {
match item {
Incoming::Event(Event::Resync(resync)) => {
// Load the state of resync.topics, then:
client.resynced(resync.round)?;
}
Incoming::Message(message) => {
// Apply it, then:
client.commit(&message.as_message());
// ...and save client.checkpoint() with the application's state.
}
_ => {}
}
}A handler works the same, and may call resynced itself: what was held comes
once it returns. Event::Continuity says how a topic’s continuity changes and
Client::topic_states() where each topic stands; without a delivery (best
effort) nothing is held, and the states and the checkpoint are kept all the
same. It is the delivery of the TypeScript and Python clients and of the Swift,
Kotlin, Java and Dart packages, natively and in the browser.
A send can wait for its topics: client.send_requiring("cancel", &order, ["orders"]) goes once each of them is continuous on the connection, and
joins the outbox with them: each time it goes, after a reconnect too, they
are checked right before it goes, so it waits for the resync, and what was
sent after it waits in its place with it. Sends that require nothing go past
one that has not joined the outbox yet. Its ttl counts from the send: one
that runs out while it waits is NotDelivered (expired). It is the user’s
who sent it and goes nowhere once the server takes a connection for another;
at most send_queue of them wait.
§The outbox
use wse_client::{Event, Incoming, SendOptions};
let mut config = wse_client::Config::new("wss://example.com/wse");
config.offline = Some(wse_client::Offline::in_file("/path/in/the/app/queue"));
let (client, mut events) = wse_client::connect(config).await?;
// With a connection or without: it waits until the server answers for it.
let mut options = SendOptions::default();
options.id = Some("order-17".to_owned());
client
.send_with("order", &serde_json::json!({"item": "tea"}), &options)
.await?;
while let Some(incoming) = events.next().await {
match incoming {
Incoming::Event(Event::Delivered(ids)) => println!("delivered: {ids:?}"),
Incoming::Event(Event::NotDelivered(not)) => {
println!("not delivered ({}): {:?}", not.reason, not.ids);
}
_ => {}
}
}Every message sent, except a live or signed one, waits in the outbox until
the server answers for it. It goes numbered and in order; after a reconnect
what was not answered goes again, before anything new, and a 3.2 server
drops a repeat of one it took. What the server cannot tell (the client
comes from a node of a cluster that is gone, or its handler panicked on the
message) reaches its application marked possible_repeat, and the
application checks the message’s id.
The application hears what became of each message by its id:
Event::Delivered, or Event::NotDelivered and why: the server refused it
(busy, rate_limited, missing, not_ready), it was not written within
its time (expired: max_age, an hour, or the ttl it was sent with), it
was the last user’s (user_changed), the application cleared the outbox
(cleared: client.clear(), for a sign-out), or the client closed with it
in memory (closed). A close waits up to a second for the server’s answers
first.
At most max_messages (100,000) and max_bytes (16 MiB) wait, the ones
written and not answered yet included: past either send waits for room,
and try_send fails at once with Error::Full. kept() says how many
wait. A send that requires topics waits in memory until they are
continuous, and joins the outbox then, with them: they are in its file too,
and checked again each time it goes.
SendOptions also names a request’s correlation_id (cid), which the
server application’s reply names back (Message::correlation_id), and its
priority (pri: 10 critical, 8 high, 5 normal, 3 low, 1 background), for
the server’s application; the outbox keeps the order messages were sent in.
With live the message goes on the connection that is ready now, or not at
all, as try_send_live sends it, with the same id, correlation id and
priority.
wse_client::connect (and Client::connect) returns once the server took
the connection, and fails when the first attempt does. For an application
that may start without a network, wse_client::start (and Client::start)
returns at once: the connection is made in the background, a failed first
attempt made again as any later one (Config::reconnect), and what is
sent meanwhile waits in the outbox; a client that gives up says so in
closed().
Offline::in_memory() (the default) keeps the outbox for the client’s
life. Offline::in_file(path) keeps it in a file, so that it outlives the
application: the file is the client’s alone while it runs (<path>.lock
beside it says so), each line is checked (a CRC-32 and a salt) and read up
to the first that does not check, the lines of a batch are written at once
and synced to the disk by a thread of its own, a message goes on the
connection once the file holds it (so that after a crash its number is
never another message’s), and the file is written anew once what went
outweighs what is kept. A file that cannot be opened
or written is an Event::StorageFailed: the outbox goes on in memory. In
the browser, Offline::in_indexed_db(name) keeps it in the IndexedDB
database name, so that it outlives the page: each tab keeps a producer of
its own there, held by a Web Lock while the tab lives, and sends on its own
connection; a page that opens takes over what a closed or crashed tab left,
under that tab’s producer, keeping each message’s number. The server
recognizes repeats while it retains that producer’s decisions. The
TypeScript client uses the same database. Web Locks need a secure
context (HTTPS, or localhost): without them, or without IndexedDB, it is an
Event::StorageFailed and the outbox stays in memory. config.offline = None has no outbox: a message goes on the connection that is ready, or the
send fails with Error::NotConnected, as before 3.2.
With a server older than 3.2, which does not answer, a message is gone once written, as before.
§In the browser
The same crate builds for wasm32-unknown-unknown, for applications in
Leptos, Yew, Dioxus and the like, with the same API. There the connection
is the browser’s WebSocket, on the page’s task: a handler need not be
Send, and Client::connect is awaited from the page’s own futures.
- The browser sends the page’s cookies with the upgrade and no header, so
the server takes the client by its cookie (
jwt_cookie_name);token,headersandtlsdo not apply, and aTokenProvideris asked before each attempt only to refresh what it must. - The browser cannot hold a WebSocket back: what comes waits in memory
until the handler or the stream takes it (
Handler::readyis not called there), and the client’s own sends wait while the browser holds a megabyte unsent. - End-to-end encryption and signed messages run on RustCrypto there, checked against the same vectors as aws-lc-rs.
§One connection for every window
With the bridge feature, bridge::Owner holds one connection for the
windows of an application (a Tauri application’s webviews, say), each of
which holds a client of the TypeScript package’s wse-client/bridge over a
channel the application carries. The owner subscribes the union of the
windows’ topics, hands each window what it holds and listens to in batches
it acknowledges (a window that falls behind holds back only itself, and
past its limits hears its topics broke off), sends the windows’ messages
through its outbox, and routes each outcome to the window that sent it,
whatever ids the windows chose. After a reconnect, a window’s send that
requires topics waits in the outbox until that window resynced them. It
connects while a window, or the application (Owner::retain), shares the
connection. The token, the keys and the outbox stay with the owner; no frame
carries them. Not in the browser. tauri-plugin-wse
is a Tauri application’s owner, ready made on it.
use wse_client::bridge::{Options, Owner, Window};
struct Webview; // the application's channel to one window
impl Window for Webview {
fn post(&self, frame: String) { /* hand the JSON frame to the window */ }
fn close(&self) {}
}
let mut config = wse_client::Config::new("wss://example.com/wse");
config.token = Some("...".to_owned());
let owner = Owner::start(config, Options::default()).await?;
let window = owner.attach(Webview);
owner.frame(window, r#"{"k":"hello","v":1,"agent":"main"}"#); // each frame the window sends§Features
aws-lc(default):wss://with rustls and the Mozilla root certificates, end-to-end encryption and signed messages, on aws-lc-rs (in the browser: encryption and signing on RustCrypto, TLS the browser’s). Without it there is no end-to-end encryption or signing, and natively nowss://; in the browserwss://is the browser’s own TLS.fips: aws-lc-rs built from its FIPS 140-3 validated module (needs CMake and Go).bridge: thebridgemodule, one connection for every window of an application (withaws-lc).native: thenativemodule the Swift, Kotlin, Java and Dart packages are built on: plain types, one shared runtime, cancellable calls, events read in bounded batches, andwss://checked by the system’s certificate verifier. Not in the browser.
The protocol is documented in the repository.
§License
MIT
Structs§
- Checkpoint
Rejected - Why the delivery’s checkpoint was not taken:
malformed,version,realm,user,consumerorschema. - Client
- A connection to a WSE server, made again when it is lost.
- Config
- What a client connects to and how it behaves. The defaults are those of
the Python and TypeScript clients. Its
Debugprint leaves the token, the headers’ values and the signing key out. - Connected
- Delivery
- The recover-or-resync delivery, for an application where every fact
counts: a topic that is not continuous (a cold start, every reconnect, a
break) holds its messages until the application brought its state up to
date (
Event::Resync, thenClient::resynced); the checkpoint is what it committed it applied, and recovery starts from it. - Disconnected
- Encryption
- End-to-end encryption: a session with the server inside the connection, so whatever terminates TLS in between (a proxy, a load balancer) sees nothing.
- Events
- What the server sends, in order. While the application does not read, the client stops reading too and the server holds back; in the browser, which cannot hold a WebSocket back, what comes waits in memory.
- Gap
- Messages of a topic that will not come.
- Message
- A message from the server, read in place from its frame: nothing is copied until the application asks for it.
- NotDelivered
- Messages of the outbox that will not be delivered, and why: the
server’s refusal (
busy,rate_limited,missing,not_ready), orexpired,user_changed,cleared. - Offline
- The client’s outbox (
Config::offline): where the messages wait until the server answers for them, and how many. - Owned
Message - A message the application keeps:
Message::to_owned, or what theEventsstream yields. - Paused
- Position
- Where a topic message stands in its topic: the epoch of the topic’s history on the server that published it, and the offset within it.
- Reconnect
- How a lost connection is made again.
- Recovery
- What a subscription replayed of a topic.
- Resync
- Topics waiting for the application to bring its state of them up to date.
- Send
Options - How a message goes (
Client::send_with). - Server
Error - An
errormessage from the server. - Signature
- What proves who sent a signed message: keep all three.
- Storage
Failed - What failed in the outbox’s file (
operation:openorwrite), and why. - Stream
Position - Where a consumer stands in one stream of a topic: the last offset it committed it applied.
- Subscribed
- Token
Provider - Fetches the token for each connection attempt, so an expired one is replaced without restarting the client. In the browser, which sends the page’s cookies itself and no header, it is asked before each attempt (to refresh the cookie, say) and what it returns is not sent.
- Topic
Continuity - A topic’s continuity, as it changed.
- Topic
State - A topic held: whether the server took it, and whether its messages run without a break.
Enums§
- Continuity
- Whether a topic’s messages run without a break.
- Error
- Why a call failed or the client stopped.
- Event
- Something that happened to the connection.
- Format
- The wire format the server sends this client.
- GapReason
- Why a gap was not filled.
- Incoming
- What the
Eventsstream yields. - Status
- Where the client stands.
- Subscription
- Whether the server took a topic.
Traits§
- Handler
- What the application does with what the server sends.
- Maybe
Send Send, except in the browser: a handler there may hold what the page has, and runs on its one thread.
Functions§
- connect
- Connects, and returns the client with the stream of what the server
sends. See
Client::connect. - generate_
key - A new ECDSA P-256 private key, PKCS#8 DER: a client’s signing key
(
Config::signing_key), or a key for a server. - public_
key - The public key of a PKCS#8 private key, base64 of the uncompressed
point: what a server binds to a connection (
set_client_key), and what clients pin. - public_
key_ jwk - The public key of a PKCS#8 private key as a JWK, for the
cnfclaim of the token a server issues the client (RFC 7800): the server then takes the client’s signed messages without being told the key. - start
- Starts, and returns at once the client with the stream of what the
server sends: the connection is made in the background. See
Client::start.