arcly-stream 0.1.1

A high-performance live-media streaming kernel: lock-free zero-copy frame fan-out, instant-start GOP cache, pluggable HLS/recording, and trait-driven protocol/storage/auth/observer extension points — runtime, config, and metrics free.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219

//! **arcly-stream** — a high-performance live-media streaming kernel.
//!
//! The pub/sub core of a streaming server: lock-free, zero-copy frame fan-out to
//! an arbitrary number of late-joining subscribers, with instant-start GOP
//! replay, publish/play authorization, live QoS, and a pluggable media plane —
//! all free of any config schema, metrics singleton, or HTTP runtime. You bring
//! the wire protocols and telemetry by implementing small traits; the kernel
//! owns the hard, hot-path part.
//!
//! ```
//! use arcly_stream::prelude::*;
//!
//! let engine = Engine::builder()
//!     .max_publishers(1000)
//!     .application(AppSpec::new("live").gop_cache(120)) // instant-start
//!     .build();
//! assert_eq!(engine.list_apps().len(), 1);
//! ```
//!
//! # Architecture
//!
//! The crate is layered as a small always-on **kernel**, a feature-gated
//! **media plane**, and **deferred wire protocols** that exist only as traits:
//!
//! ```text
//!   ┌─────────────────────── Core kernel (always on) ───────────────────────┐
//!   │  Engine · StreamHandle (broadcast fan-out, GOP cache, live QoS)        │
//!   │  PublishRegistry / PlaybackRegistry / EventBus · Observer · auth ·     │
//!   │  audit · health · MediaFrame / CodecId · StreamError                   │
//!   └───────────────────────────────────────────────────────────────────────┘
//!         ▲ implements                              ▲ feature-gated, pure Rust
//!         │                                         │
//!   ┌─────┴── Wire protocols & muxers ──┐   ┌───────┴── Media plane ──────────┐
//!   │ rtmp   RTMP publish/play (ready)  │   │ codec  H.264/H.265/AV1/VP9/VVC  │
//!   │ mpegts MPEG-TS muxer (ready)      │   │ hls    Packager + segmenter     │
//!   │ ProtocolHandler  (SRT/WHIP/RTSP)  │   │ record RecordingSink            │
//!   │ Transcoder/ClusterRelay (traits)  │   │ ingest TCP accept loop          │
//!   └───────────────────────────────────┘   └─────────────────────────────────┘
//! ```
//!
//! Two wire-plane components ship operational: the [`rtmp`](crate::protocol::rtmp)
//! handler (publish + play over real RTMP) and the
//! [`MpegTsMuxer`](crate::packager::MpegTsMuxer) (player-compatible `.ts` output).
//! Remaining protocols (SRT/WHIP/RTSP) and ABR/cluster backends are still trait
//! contracts a host implements.
//!
//! ## Design invariants
//!
//! - **Zero-copy fan-out** — frames are `bytes::Bytes`; one publish clones an
//!   `Arc<MediaFrame>` pointer to every subscriber, never the payload.
//! - **No locks on the hot path** — the frame bus is a `tokio::broadcast`
//!   channel; cached CONFIG frames and the GOP head use `ArcSwap` / atomics.
//! - **No global state** — telemetry ([`Observer`]), authorization
//!   ([`StreamAuthenticator`]), and audit ([`AuditSink`]) are *injected* traits
//!   with no-op/permit-all defaults, never singletons. The engine is
//!   `Send + Sync` and can be instantiated many times per process.
//! - **Contracts are traits** — protocols depend on [`PublishRegistry`] /
//!   [`PlaybackRegistry`], not on the concrete [`Engine`], so a host can swap in
//!   its own bus.
//! - **Pluggable ingest** — a new wire protocol is one [`InboundProtocol`]
//!   ([`inbound`]) implementation, registered with
//!   [`EngineBuilder::protocol`](crate::EngineBuilder::protocol). The engine runs
//!   any worker that satisfies the trait; the bundled `RtmpHandler` is just the
//!   first reference implementation.
//! - **`#![forbid(unsafe_code)]`** — the kernel contains no `unsafe`, enforced
//!   by the compiler.
//!
//! ## Module layout
//!
//! | Module        | Responsibility                                                |
//! |---------------|---------------------------------------------------------------|
//! | [`frame`]     | `MediaFrame`, `CodecId`, `FrameType`, `FrameFlags`            |
//! | [`identity`]  | `AppName`, `StreamId`, `StreamKey` (cheap `Arc<str>` ids)     |
//! | [`error`]     | `StreamError` and the crate [`Result`] alias                 |
//! | [`traits`]    | `MediaSource`, `MediaSink`, `ProtocolHandler`, `StorageBackend`, `HwAccelBackend` |
//! | [`bus`]       | `StreamHandle` fan-out (GOP cache, live QoS) + registry contracts |
//! | [`inbound`]   | `InboundProtocol` trait + `IngestContext` / `PublishSession` — the multi-protocol ingestion seam |
//! | [`engine`]    | `Engine`, `EngineBuilder`, `AppSpec` — composition, run loop, source pump, idle reaper |
//! | [`auth`]      | `StreamAuthenticator` publish/play authorization (permit-all default) |
//! | [`observe`]   | `Observer` telemetry hook + `NoopObserver`                   |
//! | [`audit`]     | `AuditSink` + `AuditPipeline` lifecycle audit trail          |
//! | [`health`]    | `HealthCheck` / `HealthRegistry` readiness aggregation       |
//! | [`transcode`] | `Transcoder` / `RenditionSpec` ABR contracts                |
//! | [`cluster`]   | `ClusterRelay` origin/edge federation contracts             |
//! | [`testing`]   | In-memory helpers for downstream tests                       |
//!
//! ## Cargo features
//!
//! | Feature | Effect |
//! |---|---|
//! | *(none)* | Pure engine: frame model, bus, traits, auth/audit/health/cluster contracts |
//! | `codec` | Baseline H.264 NAL/SPS + AAC ADTS parsers ([`codec`]) |
//! | `codec-h265` / `codec-av1` / `codec-vp9` / `codec-vvc` | Next-gen codec parsers (opt-in, pure Rust) |
//! | `codecs-all` | Every codec parser at once |
//! | `storage-fs` | Filesystem [`StorageBackend`] adapter |
//! | `ingest` | Shared TCP accept loop, keyframe gate, rate limiter, NAL scan |
//! | `hls` | HLS/LL-HLS packager: `Muxer`/`Packager`, segmenter, playlists ([`packager`]) |
//! | `mpegts` | Native MPEG-TS `Muxer` (`MpegTsMuxer`) — playable `.ts` segments |
//! | `rtmp` | Working RTMP publish/play [`ProtocolHandler`](protocol::rtmp::RtmpHandler) |
//! | `record` | Segment/recording sink over a [`StorageBackend`] ([`record`]) |
//! | `hls` / `record` | Shared keyframe-boundary [`segment`] timing |
//! | `metrics` | Prometheus [`Observer`] implementation |
//! | `macros` | `#[derive(MediaSink)]`, `#[protocol]` ergonomics |
//! | `full` | everything |

#![cfg_attr(docsrs, feature(doc_cfg))]
// The engine contains no `unsafe` — make that a compiler-enforced guarantee so
// a future change can't quietly introduce it.
#![forbid(unsafe_code)]

pub mod audit;
pub mod auth;
pub mod bus;
pub mod cluster;
pub mod engine;
pub mod error;
pub mod frame;
pub mod health;
pub mod identity;
pub mod inbound;
pub mod observe;
pub mod testing;
pub mod traits;
pub mod transcode;

#[cfg(feature = "_codec")]
#[cfg_attr(docsrs, doc(cfg(feature = "codec")))]
pub mod codec;

#[cfg(feature = "storage-fs")]
#[cfg_attr(docsrs, doc(cfg(feature = "storage-fs")))]
pub mod storage;

#[cfg(feature = "ingest")]
#[cfg_attr(docsrs, doc(cfg(feature = "ingest")))]
pub mod protocol;

#[cfg(feature = "hls")]
#[cfg_attr(docsrs, doc(cfg(feature = "hls")))]
pub mod packager;

#[cfg(feature = "record")]
#[cfg_attr(docsrs, doc(cfg(feature = "record")))]
pub mod record;

#[cfg(any(feature = "hls", feature = "record"))]
#[cfg_attr(docsrs, doc(cfg(any(feature = "hls", feature = "record"))))]
pub mod segment;

#[cfg(feature = "metrics")]
#[cfg_attr(docsrs, doc(cfg(feature = "metrics")))]
pub mod observability;

// ── Flat re-exports (match sc-core's surface so migration is mechanical) ────
pub use audit::{AuditAction, AuditPipeline, AuditRecord, AuditSink};
pub use auth::{AllowAll, Credentials, StreamAuthenticator};
pub use bus::{
    EventBus, PlaybackRegistry, PublishRegistry, Qos, StreamEvent, StreamEventKind, StreamHandle,
    StreamMetadata, StreamState, Subscription,
};
pub use cluster::{ClusterRelay, NodeAddr};
pub use engine::{AppSpec, Engine, EngineBuilder, EngineConfig};
pub use error::{ProtocolErrorKind, StreamError};
pub use frame::{AudioFrame, CodecId, FrameFlags, FrameType, MediaFrame, VideoFrame};
pub use health::{HealthCheck, HealthRegistry, HealthReport, HealthStatus};
pub use identity::{AppName, StreamId, StreamKey};
pub use inbound::{InboundProtocol, IngestContext, PublishSession};
pub use observe::{NoopObserver, Observer};
pub use traits::{HwAccelBackend, MediaSink, MediaSource, ProtocolHandler, StorageBackend};

// Re-export `bytes` so downstream crates (and doctests) can name `Bytes` — the
// payload type of every `MediaFrame` — without pinning a separate version.
pub use async_trait;
pub use bytes;
pub use transcode::{RenditionSpec, Transcoder};

/// The crate-wide fallible result. Mirrors `sc-core::Result`.
pub type Result<T> = std::result::Result<T, StreamError>;

/// Re-exports the derive macros expand against. Stable surface for codegen so
/// internal files can move without touching the macro crate.
#[cfg(feature = "macros")]
#[doc(hidden)]
pub mod __macro_support {
    pub use crate::bus::PublishRegistry;
    pub use crate::traits::{MediaSink, ProtocolHandler};
    pub use crate::{MediaFrame, Result, StreamError, StreamKey};
    pub use async_trait::async_trait;
    pub use std::sync::Arc;
    pub use tokio_util::sync::CancellationToken;
}

/// `use arcly_stream::prelude::*;` — everything a downstream user needs.
pub mod prelude {
    pub use crate::audit::{AuditPipeline, AuditRecord, AuditSink};
    pub use crate::auth::{AllowAll, Credentials, StreamAuthenticator};
    pub use crate::bus::{
        EventBus, PlaybackRegistry, PublishRegistry, Qos, StreamEvent, StreamEventKind,
        StreamHandle, StreamState, Subscription,
    };
    pub use crate::cluster::{ClusterRelay, NodeAddr};
    pub use crate::engine::{AppSpec, Engine, EngineBuilder, EngineConfig};
    pub use crate::error::{ProtocolErrorKind, StreamError};
    pub use crate::frame::{CodecId, FrameFlags, FrameType, MediaFrame};
    pub use crate::health::{HealthCheck, HealthRegistry, HealthStatus};
    pub use crate::identity::{AppName, StreamId, StreamKey};
    pub use crate::inbound::{InboundProtocol, IngestContext, PublishSession};
    pub use crate::observe::{NoopObserver, Observer};
    pub use crate::traits::{
        HwAccelBackend, MediaSink, MediaSource, ProtocolHandler, StorageBackend,
    };
    pub use crate::transcode::{RenditionSpec, Transcoder};
    pub use crate::Result;
    pub use async_trait::async_trait;
    pub use tokio_util::sync::CancellationToken;

    #[cfg(feature = "macros")]
    pub use arcly_stream_macros::{protocol, MediaSink as MediaSinkDerive};
}