# wse-client
The Rust client of [WSE](https://github.com/silvermpx/wse): topics with
recovery, reconnects, msgpack, end-to-end encryption and signed messages,
on tokio and in the browser (WebAssembly).
It speaks the protocol of the TypeScript and Python clients to the same
servers: [`wse`](https://crates.io/crates/wse) in Rust and
[`wse-server`](https://pypi.org/project/wse-server/) in Python.
```toml
[dependencies]
wse-client = "3.2"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
tokio = { version = "1", features = ["rt-multi-thread", "macros"] }
```
## A stream of what the server sends
```rust,no_run
use wse_client::{Config, Event, Incoming};
#[tokio::main]
async fn main() -> Result<(), wse_client::Error> {
let mut config = Config::new("wss://example.com/wse");
config.token = std::env::var("WSE_TOKEN").ok();
config.topics = vec!["prices".into()];
let (client, mut events) = wse_client::connect(config).await?;
client
.send("hello", &serde_json::json!({"from": "a bot"}))
.await?;
while let Some(incoming) = events.next().await {
match incoming {
Incoming::Message(message) => {
println!("{} {:?}: {}", message.kind(), message.topic(), message.payload_value()?);
}
// Messages that will not come: reload what depends on them.
Incoming::Event(Event::Gap(gap)) => println!("{} lost {:?}", gap.topic, gap.missed()),
Incoming::Event(event) => println!("{event:?}"),
_ => {}
}
}
Ok(())
}
```
## A handler, without copies
A `Handler` is called on the connection's own task with each message read
in place from its frame; its payload decodes into the application's types,
borrowing from the frame.
```rust,no_run
use wse_client::{Client, Config, Event, Handler, Message};
#[derive(serde::Deserialize)]
struct Tick<'a> {
symbol: &'a str,
price: f64,
}
struct Prices;
impl Handler for Prices {
fn message(&mut self, message: &Message<'_>) {
if message.kind() == "tick" {
if let Ok(tick) = message.payload::<Tick>() {
println!("{} {}", tick.symbol, tick.price);
}
}
}
fn event(&mut self, event: Event) {
if let Event::Gap(gap) = event {
println!("{} lost {:?} messages", gap.topic, gap.missed());
}
}
}
#[tokio::main]
async fn main() -> Result<(), wse_client::Error> {
let mut config = Config::new("ws://127.0.0.1:5006/wse");
config.topics = vec!["prices".into()];
let client = Client::connect(config, Prices).await?;
client.closed().await;
Ok(())
}
```
`examples/chat_client.rs` talks to the chat server of the `wse` crate.
## What a client does
- **Topics with recovery.** Each topic message carries its position; the
client drops duplicates, holds what comes past a gap while it asks the
server for what is missing, and after a reconnect subscribes again from
where it left off. What cannot be recovered is reported as a `Gap`.
- **Reconnects** with backoff and jitter, a fresh token from a
`TokenProvider` each time: a refused or expired token is fetched again,
three times in a row at most. A fixed token that is refused, a server that
refuses the session, and the application's own close end the client.
- **Heartbeats** both ways, and a connection silent for too long is made
again.
- **Wire formats.** JSON in text or binary frames, compressed frames, and
msgpack (`Format::Msgpack`, 24-41% fewer bytes).
- **End-to-end encryption** with the server's identity key pinned
(`Encryption`): whatever terminates TLS in between sees nothing.
- **Signed messages.** `Client::send_signed` signs with the client's key
for the server's; the server's signed messages are checked against the
keys the client pins, and each carries its proof. With encryption, the
client also proves its key inside the session, which binds a token that
names the key to this client (`Encryption::bind_session`, on by default),
and stops unless the server confirms the key. A signed message goes out
on the connection it was signed on, or not at all.
- **Live messages.** `Client::send_live` and `try_send_live` send on the
connection that is ready now, or not at all: one the connection did not
write is dropped, never sent on a later connection. For commands that
must not run late (a valve, a door, an order).
- **The outbox.** Every message sent, except a live or signed one, waits
until the server answers for it and goes again after a reconnect; a 3.2
server takes each once where it can tell, and marks it a possible repeat
where it cannot. The application hears what became of each. In memory, or in a file
that outlives the application (see below). Every message carries an
`id` (a UUID, or the application's own), as from the TypeScript and
Python clients.
- **Another user.** When the token provider's next token is another
user's, what the last user queued and did not send goes nowhere, and the
topics are subscribed again from the start.
- **Recovery per stream.** A topic published on several servers of a
cluster has a stream per server; a recovery names the client's offset in
each, and a stream that stayed quiet on the connection for longer than the
server keeps one is no longer named. Time away keeps every position.
- **Pause and close.** `pause(reason)` ends the connection, after what is
queued went out, and nothing connects while any reason holds;
`resume(reason)` for the last one connects at once and recovers what was
missed. `close_within(deadline)` gives what is queued the deadline to go
out and then stops.
- **Back-pressure.** A handler that waits in `ready` holds the server back;
the stream of `connect` does the same while nobody reads it.
```rust,no_run
# async fn run() -> Result<(), wse_client::Error> {
// Kept by the client. The backend names its public key in the client's
// token (`cnf`: `wse_client::public_key_jwk(&key)`).
let key = wse_client::generate_key();
let mut config = wse_client::Config::new("wss://example.com/wse");
config.token = Some("<a token whose cnf names the key>".into());
config.encryption = Some(wse_client::Encryption::pinned([
"<the server's e2e_identity_public_key>",
]));
config.signing_key = Some(key);
config.server_signing_keys = vec!["<the server's signing_public_key>".into()];
let (client, _events) = wse_client::connect(config).await?;
client
.send_signed("order", &serde_json::json!({"side": "buy", "qty": "1.5"}), Some("order-7"))
.await?;
# Ok(())
# }
```
## Where every fact counts
`Config::delivery` with a `Delivery` (recover-or-resync) holds a topic that
is not continuous -- from a cold start, after every reconnect, after a break
-- until the application brought its state of it up to date: an
`Event::Resync` asks for it, and `Client::resynced(round)` hands on what was
held, in order. After applying a message the application calls
`client.commit(&message)`; `client.checkpoint()` is what to save with its own
state and give back as `Delivery::checkpoint`, and recovery starts from it,
so a message can come twice: apply a repeat safely. A checkpoint of another
realm, user, consumer or schema is refused aloud (`Event::CheckpointRejected`),
and a message handed out before the events showed another user counts for
nobody when it is committed.
```rust,no_run
# async fn run() -> Result<(), wse_client::Error> {
use wse_client::{Delivery, Event, Incoming};
let mut config = wse_client::Config::new("wss://example.com/wse");
config.topics = vec!["orders".into()];
let mut delivery = Delivery::new("board");
delivery.checkpoint = None; // what the application saved last
config.delivery = Some(delivery);
let (client, mut events) = wse_client::connect(config).await?;
while let Some(item) = events.next().await {
match item {
Incoming::Event(Event::Resync(resync)) => {
// Load the state of resync.topics, then:
client.resynced(resync.round)?;
}
Incoming::Message(message) => {
// Apply it, then:
client.commit(&message.as_message());
// ...and save client.checkpoint() with the application's state.
}
_ => {}
}
}
# Ok(())
# }
```
A handler works the same, and may call `resynced` itself: what was held comes
once it returns. `Event::Continuity` says how a topic's continuity changes and
`Client::topic_states()` where each topic stands; without a delivery (best
effort) nothing is held, and the states and the checkpoint are kept all the
same. It is the delivery of the TypeScript and Python clients and of the Swift,
Kotlin, Java and Dart packages, natively and in the browser.
A send can wait for its topics: `client.send_requiring("cancel", &order,
["orders"])` goes once each of them is continuous on the connection, and
joins the outbox with them: each time it goes, after a reconnect too, they
are checked right before it goes, so it waits for the resync, and what was
sent after it waits in its place with it. Sends that require nothing go past
one that has not joined the outbox yet. Its `ttl` counts from the send: one
that runs out while it waits is `NotDelivered` (`expired`). It is the user's
who sent it and goes nowhere once the server takes a connection for another;
at most `send_queue` of them wait.
## The outbox
```rust,no_run
# async fn run() -> Result<(), wse_client::Error> {
use wse_client::{Event, Incoming, SendOptions};
let mut config = wse_client::Config::new("wss://example.com/wse");
config.offline = Some(wse_client::Offline::in_file("/path/in/the/app/queue"));
let (client, mut events) = wse_client::connect(config).await?;
// With a connection or without: it waits until the server answers for it.
let mut options = SendOptions::default();
options.id = Some("order-17".to_owned());
client
.send_with("order", &serde_json::json!({"item": "tea"}), &options)
.await?;
while let Some(incoming) = events.next().await {
match incoming {
Incoming::Event(Event::Delivered(ids)) => println!("delivered: {ids:?}"),
Incoming::Event(Event::NotDelivered(not)) => {
println!("not delivered ({}): {:?}", not.reason, not.ids);
}
_ => {}
}
}
# Ok(())
# }
```
Every message sent, except a live or signed one, waits in the outbox until
the server answers for it. It goes numbered and in order; after a reconnect
what was not answered goes again, before anything new, and a 3.2 server
drops a repeat of one it took. What the server cannot tell (the client
comes from a node of a cluster that is gone, or its handler panicked on the
message) reaches its application marked `possible_repeat`, and the
application checks the message's `id`.
The application hears what became of each message by its id:
`Event::Delivered`, or `Event::NotDelivered` and why: the server refused it
(`busy`, `rate_limited`, `missing`, `not_ready`), it was not written within
its time (`expired`: `max_age`, an hour, or the `ttl` it was sent with), it
was the last user's (`user_changed`), the application cleared the outbox
(`cleared`: `client.clear()`, for a sign-out), or the client closed with it
in memory (`closed`). A close waits up to a second for the server's answers
first.
At most `max_messages` (100,000) and `max_bytes` (16 MiB) wait, the ones
written and not answered yet included: past either `send` waits for room,
and `try_send` fails at once with `Error::Full`. `kept()` says how many
wait. A send that requires topics waits in memory until they are
continuous, and joins the outbox then, with them: they are in its file too,
and checked again each time it goes.
`SendOptions` also names a request's `correlation_id` (`cid`), which the
server application's reply names back (`Message::correlation_id`), and its
`priority` (`pri`: 10 critical, 8 high, 5 normal, 3 low, 1 background), for
the server's application; the outbox keeps the order messages were sent in.
With `live` the message goes on the connection that is ready now, or not at
all, as `try_send_live` sends it, with the same id, correlation id and
priority.
`wse_client::connect` (and `Client::connect`) returns once the server took
the connection, and fails when the first attempt does. For an application
that may start without a network, `wse_client::start` (and `Client::start`)
returns at once: the connection is made in the background, a failed first
attempt made again as any later one (`Config::reconnect`), and what is
sent meanwhile waits in the outbox; a client that gives up says so in
`closed()`.
`Offline::in_memory()` (the default) keeps the outbox for the client's
life. `Offline::in_file(path)` keeps it in a file, so that it outlives the
application: the file is the client's alone while it runs (`<path>.lock`
beside it says so), each line is checked (a CRC-32 and a salt) and read up
to the first that does not check, the lines of a batch are written at once
and synced to the disk by a thread of its own, a message goes on the
connection once the file holds it (so that after a crash its number is
never another message's), and the file is written anew once what went
outweighs what is kept. A file that cannot be opened
or written is an `Event::StorageFailed`: the outbox goes on in memory. In
the browser, `Offline::in_indexed_db(name)` keeps it in the IndexedDB
database `name`, so that it outlives the page: each tab keeps a producer of
its own there, held by a Web Lock while the tab lives, and sends on its own
connection; a page that opens takes over what a closed or crashed tab left,
under that tab's producer, keeping each message's number. The server
recognizes repeats while it retains that producer's decisions. The
TypeScript client uses the same database. Web Locks need a secure
context (HTTPS, or localhost): without them, or without IndexedDB, it is an
`Event::StorageFailed` and the outbox stays in memory. `config.offline =
None` has no outbox: a message goes on the connection that is ready, or the
send fails with `Error::NotConnected`, as before 3.2.
With a server older than 3.2, which does not answer, a message is gone
once written, as before.
## In the browser
The same crate builds for `wasm32-unknown-unknown`, for applications in
Leptos, Yew, Dioxus and the like, with the same API. There the connection
is the browser's WebSocket, on the page's task: a handler need not be
`Send`, and `Client::connect` is awaited from the page's own futures.
- The browser sends the page's cookies with the upgrade and no header, so
the server takes the client by its cookie (`jwt_cookie_name`); `token`,
`headers` and `tls` do not apply, and a `TokenProvider` is asked before
each attempt only to refresh what it must.
- The browser cannot hold a WebSocket back: what comes waits in memory
until the handler or the stream takes it (`Handler::ready` is not called
there), and the client's own sends wait while the browser holds a
megabyte unsent.
- End-to-end encryption and signed messages run on RustCrypto there,
checked against the same vectors as aws-lc-rs.
## One connection for every window
With the `bridge` feature, `bridge::Owner` holds one connection for the
windows of an application (a Tauri application's webviews, say), each of
which holds a client of the TypeScript package's `wse-client/bridge` over a
channel the application carries. The owner subscribes the union of the
windows' topics, hands each window what it holds and listens to in batches
it acknowledges (a window that falls behind holds back only itself, and
past its limits hears its topics broke off), sends the windows' messages
through its outbox, and routes each outcome to the window that sent it,
whatever ids the windows chose. After a reconnect, a window's send that
requires topics waits in the outbox until that window resynced them. It
connects while a window, or the application (`Owner::retain`), shares the
connection. The token, the keys and the outbox stay with the owner; no frame
carries them. Not in the browser. [`tauri-plugin-wse`](https://github.com/silvermpx/wse/blob/main/bindings/tauri/plugin/README.md)
is a Tauri application's owner, ready made on it.
```rust,ignore
use wse_client::bridge::{Options, Owner, Window};
struct Webview; // the application's channel to one window
impl Window for Webview {
fn post(&self, frame: String) { /* hand the JSON frame to the window */ }
fn close(&self) {}
}
let mut config = wse_client::Config::new("wss://example.com/wse");
config.token = Some("...".to_owned());
let owner = Owner::start(config, Options::default()).await?;
let window = owner.attach(Webview);
owner.frame(window, r#"{"k":"hello","v":1,"agent":"main"}"#); // each frame the window sends
```
## Features
- `aws-lc` (default): `wss://` with rustls and the Mozilla root
certificates, end-to-end encryption and signed messages, on aws-lc-rs (in
the browser: encryption and signing on RustCrypto, TLS the browser's).
Without it there is no end-to-end encryption or signing, and natively no
`wss://`; in the browser `wss://` is the browser's own TLS.
- `fips`: aws-lc-rs built from its FIPS 140-3 validated module (needs CMake
and Go).
- `bridge`: the `bridge` module, one connection for every window of an
application (with `aws-lc`).
- `native`: the `native` module the Swift, Kotlin, Java and Dart packages
are built on: plain types, one shared runtime, cancellable calls, events read
in bounded batches, and `wss://` checked by the system's certificate
verifier. Not in the browser.
The protocol is documented in the
[repository](https://github.com/silvermpx/wse/tree/main/docs).
## License
MIT