# RustStream
> RustStream is a Rust framework for services that subscribe to message brokers and publish to them. The core crate `ruststream` holds the traits, the router runtime, the codecs and an in-memory broker. Each broker (NATS, Redis, RabbitMQ, Kafka, AMQP 1.0, Google Cloud Pub/Sub, SQS / SNS, Pulsar, MQTT 5, ZeroMQ, stream files, Kinesis) is a crate of its own.
A handler is an `async fn` under `#[subscriber(..)]`, mounted on a broker with `b.include(handler)`. It takes `&T` for one message or `&[T]` for a batch. A service imports `ruststream::prelude::*` or its broker crate's prelude. The codec is a compile-time type chosen at the mount site; JSON is the default. Misuse is rejected at compile time, and the compiler error names the fix.
This site holds the entry pages. How each topic is done lives in the module overviews on docs.rs, and a broker's own settings live in its crate's documentation.
## Guides
- [Home](https://powersemmi.github.io/ruststream/latest/): what RustStream does and its design principles
- [Guides](https://powersemmi.github.io/ruststream/latest/guides/): the path from an empty crate to a running service
- [Installation](https://powersemmi.github.io/ruststream/latest/guides/getting-started/installation/): the crate, its cargo features and a broker crate
- [Quick start](https://powersemmi.github.io/ruststream/latest/guides/getting-started/quickstart/): a running service scaffolded with `cargo generate`
- [Tutorial](https://powersemmi.github.io/ruststream/latest/guides/getting-started/tutorial/): an orders service with a message type, a handler, a reply and a router, on the in-memory broker
- [Build a service, step by step](https://powersemmi.github.io/ruststream/latest/guides/build-a-service/): eleven chapters that grow one shop service on the in-memory broker, from its first subscriber to NATS
- [The service and its first subscriber](https://powersemmi.github.io/ruststream/latest/guides/build-a-service/service/): `#[subscriber]`, `include` and `#[ruststream::app]`
- [Messages: payloads, headers, codecs](https://powersemmi.github.io/ruststream/latest/guides/build-a-service/messages/): a payload type, a header contract read with `Headers<T>`, and a codec per topic
- [Replies and publishing to other topics](https://powersemmi.github.io/ruststream/latest/guides/build-a-service/replies/): a reply, an `Out` slot, and a router that binds both
- [Settlement: retry, retry_after, dead letters](https://powersemmi.github.io/ruststream/latest/guides/build-a-service/settlement/): an outcome per failure, the attempt cap and the dead-letter topic
- [Middleware and observability](https://powersemmi.github.io/ruststream/latest/guides/build-a-service/middleware/): `TracingLayer`, a `Layer` of your own, and the app that mounts them
- [Concurrency: workers and threads](https://powersemmi.github.io/ruststream/latest/guides/build-a-service/concurrency/): `threads(n)` for a handler that computes, `workers(n)` for one that waits on I/O
- [Shared state and context](https://powersemmi.github.io/ruststream/latest/guides/build-a-service/state/): state opened in `on_startup`, read as `State<T>`, and the per-delivery `Context`
- [Testing the service with TestApp](https://powersemmi.github.io/ruststream/latest/guides/build-a-service/testing/): the production app in process, a paused clock, and assertions on subscribers and topics
- [The AsyncAPI document](https://powersemmi.github.io/ruststream/latest/guides/build-a-service/asyncapi/): `asyncapi gen`, `AppInfo`, and a document test shared by both forms
- [Running beside an HTTP server, outbox](https://powersemmi.github.io/ruststream/latest/guides/build-a-service/http/): `start` beside axum, and a transactional outbox relayed to the broker
- [Switching to a networked broker](https://powersemmi.github.io/ruststream/latest/guides/build-a-service/networked-broker/): the same shop on NATS: the broker type changes, the subscribers and tests do not
## Components
- [Application](https://powersemmi.github.io/ruststream/latest/components/application/): the `RustStream` value, `#[ruststream::app]`, lifecycle hooks, running the service and the `ruststream` command
- [State](https://powersemmi.github.io/ruststream/latest/components/application/state/): shared application state, and dependency injection containers wired into it
- [Testing](https://powersemmi.github.io/ruststream/latest/components/application/testing/): `TestApp`, the test client that runs a service's production app in process, one page per function
- [Starting and stopping](https://powersemmi.github.io/ruststream/latest/components/application/testing/starting-and-stopping/): `TestApp::start`, `start_live`, `with_state`, `shutdown`
- [Driving](https://powersemmi.github.io/ruststream/latest/components/application/testing/driving/): `broker::<B>()`, `broker_named`, `message`, `publish`, `settle`, `advance`, `drain`
- [Service state](https://powersemmi.github.io/ruststream/latest/components/application/testing/service-state/): `run_result`, `assert_running`, `assert_shut_down`
- [Records](https://powersemmi.github.io/ruststream/latest/components/application/testing/records/): `subscriber(name)`, `published::<T>(name)`, `out::<M>()`
- [Subscriber assertions](https://powersemmi.github.io/ruststream/latest/components/application/testing/subscriber-assertions/): what a handler received and how it settled
- [Published assertions](https://powersemmi.github.io/ruststream/latest/components/application/testing/published-assertions/): what was published, with its headers and per-message settings
- [Subscribers](https://powersemmi.github.io/ruststream/latest/components/subscribers/): `#[subscriber]`, subscription sources, mount-site settings, batches, start positions, routers, the retry cap and destination, failure policy, typed headers
- [Messages](https://powersemmi.github.io/ruststream/latest/components/subscribers/messages/): what a handler returns: ack, retry, drop, `retry_after`, `and_after` and post-settle hooks
- [Context and extractors](https://powersemmi.github.io/ruststream/latest/components/subscribers/context/): the per-delivery `Context`, `Ctx<K>` and custom extractors
- [Codecs](https://powersemmi.github.io/ruststream/latest/components/subscribers/codecs/): where a mounting's codec comes from, JSON, CBOR, MessagePack, a codec of your own, Protobuf and the byte lanes
- [Dispatch](https://powersemmi.github.io/ruststream/latest/components/subscribers/dispatch/): `workers(n)` for handlers that wait on I/O, `threads(n)` for handlers that compute, and the `poll-diagnostics` feature that finds them
- [Middleware](https://powersemmi.github.io/ruststream/latest/components/subscribers/middleware/): `Layer` around handlers, `PublishLayer` around publishes, and the chains assembled at run time, `DynStack` and `PublishDynStack`
- [Publishing](https://powersemmi.github.io/ruststream/latest/components/subscribers/publishing/): replies, `Out` slots, where a message type goes, the publish builder, per-message settings and another broker's publisher
- [Publishers](https://powersemmi.github.io/ruststream/latest/components/subscribers/publishing/publishers/): the publisher capabilities, one page each: `Publisher`, `TransactionalPublisher`, `OwnedTransactions`, `RequestReply`
- [Transforms](https://powersemmi.github.io/ruststream/latest/components/subscribers/publishing/transforms/): `PublishTransform` on a reply or a slot, naming transforms that set the destination, and batch reply transforms
- [Observability](https://powersemmi.github.io/ruststream/latest/components/observability/): console logging, `TracingLayer`, Prometheus metrics, W3C trace propagation and OpenTelemetry export
- [AsyncAPI](https://powersemmi.github.io/ruststream/latest/components/asyncapi/): the generated AsyncAPI document, one page per function or capability
- [AsyncAPI generation](https://powersemmi.github.io/ruststream/latest/components/asyncapi/generation/): `build_spec`, JSON and YAML, `asyncapi gen`, the viewer, payload schemas and `#[derive(MessageInfo)]`
- [What the AsyncAPI document reports](https://powersemmi.github.io/ruststream/latest/components/asyncapi/reports/): replies, channel parameters, header schemas, content types and the retry extension
- [AsyncAPI servers](https://powersemmi.github.io/ruststream/latest/components/asyncapi/servers/): servers from the brokers, declared servers, security schemes and protocol bindings
- [AsyncAPI service info](https://powersemmi.github.io/ruststream/latest/components/asyncapi/service-info/): the `info` section: description, contact, license, tags, external docs and the service id
- [Memory broker](https://powersemmi.github.io/ruststream/latest/components/memory/): `MemoryBroker`, the broker inside one process, its prelude and capabilities
- [Retention](https://powersemmi.github.io/ruststream/latest/components/memory/retention/): a retaining in-memory broker that keeps the newest messages of every name for replay
- [Patterns](https://powersemmi.github.io/ruststream/latest/components/memory/patterns/): `MemoryPattern` in the NATS subject syntax, and the `Routing` rule that picks every match or the most specific one
## Module overviews on docs.rs
- [Subscribers](https://docs.rs/ruststream/latest/ruststream/runtime/index.html#subscribers): handlers, subscription sources, mount-site settings, broker steps that read them, batches, values a broker's deliveries already hold, workers, dedicated threads for handlers that compute, finding handlers that compute (the `poll-diagnostics` feature), delayed redelivery and its cap
- [Publishing](https://docs.rs/ruststream/latest/ruststream/runtime/index.html#publishing): replies, `Out` slots, publish transforms and the publish pipeline
- [Routing](https://docs.rs/ruststream/latest/ruststream/runtime/index.html#routing): routers, per-router codecs and layers
- [Context and state](https://docs.rs/ruststream/latest/ruststream/runtime/index.html#context-and-state): the per-delivery context and the shared application state
- [Typed headers](https://docs.rs/ruststream/latest/ruststream/runtime/index.html#typed-headers): headers read and written as serde types
- [Lifecycle](https://docs.rs/ruststream/latest/ruststream/runtime/index.html#lifecycle): startup and shutdown hooks, the first publish after startup
- [Middleware](https://docs.rs/ruststream/latest/ruststream/runtime/index.html#middleware): consume and publish layers
- [Failure policy](https://docs.rs/ruststream/latest/ruststream/runtime/index.html#failure-policy): what a panic or an undecodable payload does to a delivery
- [Running beside another server](https://docs.rs/ruststream/latest/ruststream/runtime/index.html#running-beside-another-server): `start()` and the running handle
- [Codecs](https://docs.rs/ruststream/latest/ruststream/codec/index.html): JSON, CBOR and MessagePack, where a codec is chosen, and payloads that carry their own bytes
- [Testing](https://docs.rs/ruststream/latest/ruststream/testing/index.html): testing a service's production app with `TestApp`, every broker in process, addressed by its production type
- [Testing against a running broker](https://docs.rs/ruststream/latest/ruststream/testing/index.html#against-a-running-broker): `TestApp::start_live`, the same test body against a stand
- [AsyncAPI](https://docs.rs/ruststream/latest/ruststream/asyncapi/index.html): the generated AsyncAPI document and its viewer
- [OpenTelemetry](https://docs.rs/ruststream/latest/ruststream/otel/index.html): trace propagation, OTLP export and per-handler metrics
- [Prometheus metrics](https://docs.rs/ruststream/latest/ruststream/metrics/index.html): consume and publish metrics
- [Logging](https://docs.rs/ruststream/latest/ruststream/logging/index.html): console logging driven by `RUST_LOG`
- [Command line](https://docs.rs/ruststream/latest/ruststream/runtime/cli/index.html): the entry point `#[ruststream::app]` generates and the `ruststream` command
## Brokers
- [Brokers](https://powersemmi.github.io/ruststream/latest/brokers/): the broker crates, and moving a service to another broker
- [In-memory broker](https://powersemmi.github.io/ruststream/latest/brokers/memory/): the broker inside one process, and the reference implementation of the contract
- [NATS](https://docs.rs/ruststream-nats): `ruststream-nats`, Core NATS and JetStream
- [Redis / Valkey](https://docs.rs/ruststream-fred): `ruststream-fred`, streams, lists and Pub/Sub
- [RabbitMQ](https://docs.rs/ruststream-lapin): `ruststream-lapin`, AMQP 0.9.1
- [Kafka](https://docs.rs/ruststream-rdkafka): `ruststream-rdkafka`, with the schema registry
- [AMQP 1.0](https://docs.rs/ruststream-amqp): `ruststream-amqp`
- [Google Cloud Pub/Sub](https://docs.rs/ruststream-gcp-pubsub): `ruststream-gcp-pubsub`
- [AWS SQS / SNS](https://docs.rs/ruststream-sqs-sns): `ruststream-sqs-sns`
- [Apache Pulsar](https://docs.rs/ruststream-pulsar): `ruststream-pulsar`
- [MQTT 5](https://docs.rs/ruststream-rumqttc): `ruststream-rumqttc`
- [ZeroMQ](https://docs.rs/ruststream-zeromq): `ruststream-zeromq`
- [Stream files / stdio](https://docs.rs/ruststream-sea-file): `ruststream-sea-file`
- [AWS Kinesis](https://docs.rs/ruststream-kinesis): `ruststream-kinesis`
- [SQL databases through sqlx](https://docs.rs/ruststream-sqlx): `ruststream-sqlx`, a transactional outbox over any broker, and task queues in Postgres, MySQL/MariaDB and SQLite tables, with statements checked at startup or at compile time
## Architecture
- [Architecture](https://powersemmi.github.io/ruststream/latest/architecture/): how RustStream is built and what its design costs a service
- [Design](https://powersemmi.github.io/ruststream/latest/architecture/design/): the broker-agnostic core, the traits a broker crate implements, and where broker settings live
- [Path](https://powersemmi.github.io/ruststream/latest/architecture/path/): the stages of one delivery, from the broker's stream through layers, decode, handler and reply to its settlement
- [Async](https://powersemmi.github.io/ruststream/latest/architecture/async/): the Tokio runtime, the tasks of each dispatch form (`workers(n)`, `threads(n)`), one delivery's timeline and the order of a graceful shutdown
- [Lifecycle](https://powersemmi.github.io/ruststream/latest/architecture/lifecycle/): the broker's states as types, `new`, `connect(self)`, `shutdown(self)`, and the app's start and shutdown
- [Settlement](https://powersemmi.github.io/ruststream/latest/architecture/settlement/): how a delivery is settled: outcomes, failure policies, retries, the attempt cap and the dead letter
- [Guarantees](https://powersemmi.github.io/ruststream/latest/architecture/guarantees/): the mistakes that do not compile, with the compiler's message for each
- [Cost](https://powersemmi.github.io/ruststream/latest/architecture/cost/): the framework's instructions and allocations per message, and where the cost goes
- [Benchmarks](https://powersemmi.github.io/ruststream/latest/architecture/benchmarks/): what the framework costs over a raw client, and the cost of its own code
- [Benchmarks: Numbers](https://powersemmi.github.io/ruststream/latest/architecture/benchmarks/numbers/): what each published number measures, and which numbers compare
- [Benchmarks: Methodology](https://powersemmi.github.io/ruststream/latest/architecture/benchmarks/methodology/): how every crate measures itself, the three loops, the run, the report and the code measurement
- [Benchmarks: Publishing](https://powersemmi.github.io/ruststream/latest/architecture/benchmarks/publishing/): the JSON document a crate publishes its results in, at a stable path
## Broker authors
- [Writing a broker](https://docs.rs/ruststream/latest/ruststream/trait.Broker.html): the `Broker` trait is the hub of the contract a broker crate implements, linking every trait, the lazy-startup ladder and the in-process mode
- [Conformance](https://docs.rs/ruststream/latest/ruststream/conformance/index.html): the suites that check a broker against the contract
- [Templates](https://docs.rs/ruststream/latest/ruststream/runtime/cli/index.html#templates-a-broker-crate-ships): the `cargo generate` template a broker crate ships
- [A worked broker crate](https://docs.rs/ruststream-nats/latest/ruststream_nats/): the NATS broker, a complete implementation of the contract
## Optional
- [API reference](https://docs.rs/ruststream): the full rustdoc of the core crate
- [Source](https://github.com/powersemmi/ruststream): the core repository