flyby 0.1.0

A high-performance Rust framework for composable data-ingestion pipelines.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114

# FlyBy

[![CI](https://github.com/thanos/flyby/actions/workflows/ci.yml/badge.svg)](https://github.com/thanos/flyby/actions/workflows/ci.yml)
[![crates.io](https://img.shields.io/crates/v/flyby.svg)](https://crates.io/crates/flyby)
[![docs.rs](https://docs.rs/flyby/badge.svg)](https://docs.rs/flyby)
[![Coverage Status](https://coveralls.io/repos/github/thanos/flyby/badge.svg?branch=main)](https://coveralls.io/github/thanos/flyby?branch=main)
[![License: MIT OR Apache-2.0](https://img.shields.io/badge/license-MIT%20OR%20Apache--2.0-blue.svg)](LICENSE-MIT)
[![MSRV](https://img.shields.io/badge/MSRV-1.95-blue)](https://blog.rust-lang.org/2026/04/16/Rust-1.95.0/)

A high-performance Rust framework for building composable
data-ingestion pipelines.

```text
Source -> Decode -> Transform -> Route -> Sink
```

Shared memory is the first production sink, but it is **not** the
defining abstraction of the project. FlyBy aims to make advanced Linux
I/O technologies (AF_XDP, io_uring, DPDK, SPDK) approachable while
preserving high performance, safety, and excellent documentation.

## Status

First public release: **0.1.0**. See [`CHANGELOG.md`](CHANGELOG.md).

| Area | Status |
|------|--------|
| `flyby::core` traits / errors | Implemented (contract layer) |
| Shared-memory SPSC sink | **Implemented** |
| Network simulator (`flyby::net`) | **Implemented** |
| Product simulator (`flyby-simulator`) | **Implemented** (CLI, TUI, FlyScenario DSL) |
| File source + framing + replay engine | **Implemented** |
| Facade builder `.run()` | Skeleton (config validation) |
| `SimplePipeline` + runtime | **Implemented** |
| Facade `run_demo()` | Builds `SimplePipeline` (sim → memory) |
| AF_XDP / DPDK / io_uring / SPDK | Stubs (`NotImplemented`) |

## Workspace layout

```text
flyby/
├── Cargo.toml
├── crates/
│   └── flyby/            # Single public crate (core, memory, net, storage, runtime)
├── examples/             # Runnable examples
├── benches/              # Criterion benchmarks
├── simulator/            # First-class simulator (flyby-sim CLI + TUI + DSL)
├── scenarios/            # FlyScenario tutorial files (*.fly.toml)
├── articles/             # Medium reproduction catalog
├── docs/                 # Project guide (mdBook) + ADRs
├── .github/workflows/    # CI + Coveralls + crates.io release
├── Dockerfile            # Linux dev container
└── .devcontainer/        # VS Code / Codespaces config
```

## Quick start

```sh
cargo build --workspace
cargo run -p flyby-examples --bin hello_pipeline

# Product simulator (no privileged networking required)
cargo run -p flyby-simulator --bin flyby-sim -- tui constant_rate
cargo run -p flyby-simulator --bin flyby-sim -- run scenarios/constant_rate.fly.toml
```

## Checks

```sh
cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
cargo doc --workspace --no-deps
```

## Feature flags

| Feature      | Default | Backend                            |
|--------------|---------|------------------------------------|
| `memory`     | yes     | In-process shared-memory sink.     |
| `af_xdp`     | no      | AF_XDP source stub (Linux).        |
| `dpdk`       | no      | DPDK source stub.                  |
| `io_uring`   | no      | io_uring storage stub.             |
| `spdk`       | no      | SPDK storage stub.                 |
| `simulator`  | no      | Builder selector for net simulator.|
| `benchmarks` | no      | Reserved (benches package always builds). |

Portable file + net-sim APIs always compile. Heavy stubs are opt-in.

## Developer container

```sh
docker build -t flyby-dev -f Dockerfile .
docker run --rm -it -v "$PWD":/workspace -w /workspace flyby-dev
```

VS Code / Codespaces users can open the repo in the configured
`.devcontainer/`.

## Documentation

- **Project guide:** `mdbook serve docs/ --open` (see [`docs/README.md`](docs/README.md))
- **Engineering standards:** [`docs/src/engineering.md`](docs/src/engineering.md) ·
  [Testing](docs/src/testing.md) · [Release](docs/src/release.md)
- **Simulator:** [`docs/src/simulator.md`](docs/src/simulator.md) ·
  [FlyScenario DSL](docs/src/scenario-dsl.md) · [Runtime](docs/src/runtime.md)
- **Contributing:** [`CONTRIBUTING.md`](CONTRIBUTING.md) · [`SECURITY.md`](SECURITY.md)
- **API reference:** `cargo doc --workspace --open`
- **Architecture decisions:** [`docs/src/adr/`](docs/src/adr/)
- **Changelog:** [`CHANGELOG.md`](CHANGELOG.md)

## License

Dual-licensed under [MIT](LICENSE-MIT) or [Apache-2.0](LICENSE-APACHE), at your option.