Expand description
The Rust client library for writing streaming applications with Fluvio
Fluvio is a high performance, low latency data streaming platform built for developers.
When writing streaming applications, two of your core behaviors are producing messages and consuming messages. When you produce a message, you send it to a Fluvio cluster where it is recorded and saved for later usage. When you consume a message, you are reading a previously-stored message from that same Fluvio cluster. Let’s get started with a quick example where we produce and consume some messages.
Examples
Fluvio’s documentation provide a set of examples for Rust. You can visit the examples page following this link.
Fluvio Echo
The easiest way to see Fluvio in action is to produce some messages and to consume them right away. In this sense, we can use Fluvio to make an “echo service”.
All messages in Fluvio are sent in a sort of category called a Topic
. You can think
of a Topic as a named folder where you want to store some files, which would be your
messages. If you’re familiar with relational databases, you can think of a Topic as
being similar to a database table, but for streaming.
As the application developer, you get to decide what Topics you create and which
messages you send to them. We need to set up a Topic before running our code. For the
echo example, we’ll call our topic echo
.
Example
The easiest way to create a Fluvio Topic is by using the Fluvio CLI.
$ fluvio topic create echo
topic "echo" created
There are convenience methods that let you get up-and-started quickly using default
configurations. Later if you want to customize your setup, you can directly use the
Fluvio
client object.
use std::time::Duration;
use fluvio::{Offset, FluvioError, RecordKey};
use futures::StreamExt;
async_std::task::spawn(produce_records());
if let Err(e) = async_std::task::block_on(consume_records()) {
println!("Error: {}", e);
}
async fn produce_records() -> anyhow::Result<()> {
let producer = fluvio::producer("echo").await?;
for i in 0..10u8 {
producer.send(RecordKey::NULL, format!("Hello, Fluvio {}!", i)).await?;
async_std::task::sleep(Duration::from_secs(1)).await;
}
// fluvio batches records by default, so call flush() when done producing to ensure all
// records are sent
producer.flush().await?;
Ok(())
}
async fn consume_records() -> anyhow::Result<()> {
let consumer = fluvio::consumer("echo", 0).await?;
let mut stream = consumer.stream(Offset::beginning()).await?;
while let Some(Ok(record)) = stream.next().await {
let key_str = record.get_key().map(|key| key.as_utf8_lossy_string());
let value_str = record.get_value().as_utf8_lossy_string();
println!("Got record: key={:?}, value={}", key_str, value_str);
}
Ok(())
}
Re-exports
pub use config::FluvioConfig;
pub use consumer::PartitionConsumer;
pub use consumer::ConsumerConfig;
pub use consumer::MultiplePartitionConsumer;
pub use consumer::PartitionSelectionStrategy;
Modules
- re-export metadata from sc-api
Structs
- An interface for interacting with Fluvio streaming
- An interface for managing a Fluvio cluster
- Output of
TopicProducer::send
Used to wait theRecordMetadata
of the record being sent. SeeFutureRecordMetadata::wait
- Describes the location of an event stored in a Fluvio partition
- Struct returned by of TopicProduce::send call, it is used to gather the record metadata associated to each send call.
- A key for determining which partition a record should be sent to.
- Metadata of a record send to a topic
- Defines parameters of retries in
DeliverySemantic::AtLeastOnce
delivery semantic. - The request payload when using a Consumer SmartModule.
- An interface for producing events to a particular topic
- Options used to adjust the behavior of the Producer. Create this struct with
TopicProducerConfigBuilder
. - Builder for
TopicProducerConfig
.
Enums
- The compression algorithm used to compress and decompress records in fluvio batches
- Defines guarantees that Producer must follow delivering records to SPU.
- Possible errors that may arise when using Fluvio
- Strategy of delays distribution.
- Indicates the type of SmartModule as well as any special data required
Traits
- A trait for defining a partitioning strategy for key/value records.
Functions
- Creates a producer that receives events from the given topic and partition
- Creates a producer that sends records to the named topic