flyby 0.1.0

A high-performance Rust framework for composable data-ingestion pipelines.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

//! The [`Sink`] trait: terminal destinations.
//!
//! A sink sits at the tail of a FlyBy pipeline. Shared memory is the
//! first production sink, but the abstraction is deliberately generic so
//! that Arrow Flight, Kafka, files, or future transports can be added
//! without changing the pipeline shape.

use super::{Lifecycle, Message, Result};

/// A terminal destination for decoded messages.
///
/// Implementations are responsible for any serialization required before
/// the message leaves the pipeline. Sinks must respect back-pressure:
/// [`Sink::write`] may return [`crate::ErrorKind::BackPressure`] when the
/// destination is temporarily full (retry), or [`crate::ErrorKind::Sink`]
/// for hard failures.
///
/// [`Lifecycle::shutdown`] should flush buffered data unless the
/// implementation documents otherwise.
pub trait Sink: Lifecycle {
    /// The message type accepted by this sink.
    type Message: Message;

    /// Write a single message to the sink.
    fn write(&mut self, message: &Self::Message) -> Result<()>;

    /// Flush any buffered messages downstream.
    ///
    /// Default is a no-op; buffering sinks override this.
    fn flush(&mut self) -> Result<()> {
        Ok(())
    }
}