flyby 0.1.1

A high-performance Rust framework for composable data-ingestion pipelines.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

//! # flyby::net
//!
//! Networking backends for FlyBy: AF_XDP, DPDK, and the in-process simulator.
//!
//! ## Architecture
//!
//! ```text
//! NIC → Backend Adapter → RawBatch → Decode → Placement → Sink
//!        (af_xdp | dpdk | sim)
//! ```
//!
//! The networking subsystem owns **packet acquisition only**. It does not
//! own application protocol parsing, shared-memory layout, placement
//! semantics, enrichment logic, or consumer behaviour.
//!
//! ## Feature flags
//!
//! | Feature | Enables |
//! |---------|---------|
//! | *(always)* | [`NetworkSource`](crate::net::NetworkSource), [`RawBatch`](crate::net::RawBatch), [`SimulatedNetSource`](crate::net::SimulatedNetSource), config types |
//! | `net`   | Parent feature (no-op today; reserved for optional gating) |
//! | `af_xdp` | [`AfXdpSource`](crate::net::AfXdpSource) stub (implies `net`) |
//! | `dpdk`  | [`DpdkSource`](crate::net::DpdkSource) stub (implies `net`) |
//!
//! Portable sim/batch types always compile. Heavy bindings are never default.
//!
//! ## Hardware requirements
//!
//! The simulator works on any platform. AF_XDP and DPDK require a real
//! Linux host with specific kernel versions and NIC drivers. Docker Desktop
//! on macOS and standard GitHub-hosted CI are **not** sufficient for
//! hardware-backed networking tests. See the module documentation for each
//! backend for the full requirements list.
//!
//! ## Backpressure
//!
//! Networking sources must not drop packets without accounting. Drops are
//! tracked via [`RawBatch::record_drop`](crate::net::RawBatch::record_drop) / [`RawBatch::dropped`](crate::net::RawBatch::dropped) and should
//! be exposed via [`NetMetricKey`](crate::net::NetMetricKey). Oversized frames may be truncated with
//! [`PacketMeta::original_len`](crate::net::PacketMeta::original_len) preserved; prefer sizing slots correctly.

#![forbid(unsafe_code)]
#![deny(missing_docs)]
#![deny(rustdoc::broken_intra_doc_links)]

pub mod batch;
pub mod config;
pub mod metrics;
pub mod sim;
pub mod source;

#[cfg(feature = "af_xdp")]
pub mod af_xdp;

#[cfg(feature = "dpdk")]
pub mod dpdk;

// ---------------------------------------------------------------------------
// Flat re-exports for the common case: `use crate::net::*`
// ---------------------------------------------------------------------------

pub use batch::{PacketMeta, PushResult, RawBatch};
pub use config::{AfXdpConfig, DpdkConfig, SimNetConfig, UmemConfig, XdpConfig, XdpMode};
pub use metrics::NetMetricKey;
pub use sim::SimulatedNetSource;
pub use source::{BackpressurePolicy, NetworkSource};

#[cfg(feature = "af_xdp")]
pub use af_xdp::AfXdpSource;

#[cfg(feature = "dpdk")]
pub use dpdk::DpdkSource;