spate-core 0.2.0

Engine for the Spate framework: records, operator chains, source/sink abstractions, checkpointing, backpressure, config, metrics, and the pipeline runtime. Applications should depend on the `spate` facade crate instead.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

//! The engine of the Spate framework.
//!
//! `spate-core` contains the pipeline runtime and every technology-neutral
//! abstraction: records and their checkpoint tokens, the operator chain,
//! the source and sink traits, checkpointing, backpressure, configuration
//! loading, metrics, and the admin server.
//!
//! Applications should depend on the [`spate`](https://crates.io/crates/spate)
//! facade crate rather than on `spate-core` directly.
//!
//! The properties the engine is arranged around are published as
//! [Invariants], the decisions behind them as [Decisions], and the metric
//! taxonomy as [Metrics]. Module documentation below cites the first two by
//! identifier alone (`INV-2`, `ADR-0013`).
//!
//! [Invariants]: https://spate.kainth.dev/docs/INVARIANTS
//! [Decisions]: https://spate.kainth.dev/docs/adr/
//! [Metrics]: https://spate.kainth.dev/docs/METRICS

// tokio's own sources change shape under `--cfg loom` (net disappears), so
// anything touching tokio::net is compiled out of loom model builds.
#[cfg(not(loom))]
pub mod admin;
/// Re-export of the [`bytes`] crate: [`RowEncoder`](sink::RowEncoder)
/// signatures take [`bytes::BytesMut`], so connector authors can use this
/// re-export instead of declaring their own `bytes` dependency (declaring
/// one is also fine; versions are compatible per the workspace pin).
pub use bytes;

pub mod backpressure;
pub mod checkpoint;
// Resolves the reserved `chunk:` block into `ops::ChunkConfig`, so it follows
// `ops` out of loom model builds (nothing loom-modelled reads config).
#[cfg(not(loom))]
pub mod config;
// References `source` types, so it shares the source module's loom gate.
#[cfg(not(loom))]
pub mod coordination;
pub mod deser;
pub mod error;
pub mod framing;
pub mod metrics;
#[cfg(not(loom))]
pub mod ops;
#[cfg(not(loom))]
pub mod pipeline;
pub mod record;
#[cfg(not(loom))]
pub mod sink;
#[cfg(not(loom))]
pub mod source;
pub mod telemetry;

pub use error::{DeserError, ErrorClass, ErrorPolicy, FatalError, SinkError, SourceError};
pub use framing::FramingContract;
pub use record::{Flow, PartitionId, RawPayload, Record, RecordMeta};