Crate sea_streamer_socket

sea-streamer-socket: Backend-agnostic Socket API

Akin to how SeaORM allows you to build applications for different databases, SeaStreamer allows you to build stream processors for different streaming servers.

While the sea-streamer-types crate provides a nice trait-based abstraction, this crates provides a concrete-type API, so that your program can stream from/to any SeaStreamer backend selected by the user on runtime.

This allows you to do neat things, like generating data locally and then stream them to Redis / Kafka. Or in the other way, sink data from server to work on them locally. All without recompiling the stream processor.

If you only ever work with one backend, feel free to depend on sea-streamer-redis / sea-streamer-kafka directly.

A small number of cli programs are provided for demonstration. Let’s set them up first:

# The `clock` program generate messages in the form of `{ "tick": N }`
alias clock='cargo run --package sea-streamer-stdio  --features=executables --bin clock'
# The `relay` program redirect messages from `input` to `output`
alias relay='cargo run --package sea-streamer-socket --features=executables,backend-kafka,backend-redis --bin relay'

Here is how to stream from Stdio ➡️ Redis / Kafka. We generate messages using clock and then pipe it to relay, which then streams to Redis / Kafka:

# Stdio -> Redis
clock -- --stream clock --interval 1s | \
relay -- --input stdio:///clock --output redis://localhost:6379/clock
# Stdio -> Kafka
clock -- --stream clock --interval 1s | \
relay -- --input stdio:///clock --output kafka://localhost:9092/clock

Here is how to stream between Redis ↔️ Kafka:

# Redis -> Kafka
relay -- --input redis://localhost:6379/clock --output kafka://localhost:9092/clock
# Kafka -> Redis
relay -- --input kafka://localhost:9092/clock --output redis://localhost:6379/clock

Here is how to replay the stream from Kafka / Redis:

relay -- --input redis://localhost:6379/clock --output stdio:///clock --offset start
relay -- --input kafka://localhost:9092/clock --output stdio:///clock --offset start

Structs

• SeaConnectOptions
  sea-streamer-socket concrete type of ConnectOptions.
• SeaConsumer
  sea-streamer-socket concrete type of Consumer.
• SeaConsumerOptions
  sea-streamer-socket concrete type of ConsumerOptions.
• SeaProducer
  sea-streamer-socket concrete type of Producer.
• SeaProducerOptions
  sea-streamer-socket concrete type of ProducerOptions.
• SeaStreamer
  sea-streamer-socket concrete type of Streamer.

Enums

• Backend
  sea-streamer-socket Enum for identifying the underlying backend.
• BackendErr
  sea-streamer-socket the concrete backend error.
• NextFuture
  sea-streamer-socket concrete type of Future that will yield the next message.
• SeaMessage
  sea-streamer-socket concrete type of Message.
• SeaMessageStream
  sea-streamer-socket concrete type of Stream that will yield the next messages.
• SeaStreamReset
• SendFuture
  sea-streamer-socket concrete type of a Future that will yield a send receipt.

Traits

• SeaStreamerBackend
  sea-streamer-socket methods shared by Sea* types.

Type Definitions

• Error
  sea-streamer-socket the concrete error type.