media_plane/trunk.rs
1//! [`Trunk`] — the sample ring, [`TrunkWriter`], and [`SampleCursor`] (plan
2//! step 3b-i); the segment log, [`SegmentCursor`], and the
3//! lossless-by-retention pinning mechanism (plan step 3b-ii); the 90 kHz
4//! event log, [`EventCursor`], and [`EventAnchor`] (plan step 3b-iii); the
5//! live-part log and the [`Trunk::listen`] reader-wake primitive (plan step
6//! 3b-iv), closing the two gaps step 3d found while reading
7//! `hls-runtime/src/server/` before writing the egress traits — see
8//! [The live-part log](#the-live-part-log-parts-before-their-segment-closes)
9//! and
10//! [The reader-wake primitive](#the-reader-wake-primitive-listen-not-one-registration-per-remote-peer)
11//! below; and now [`SegmentWriter`], splitting the single write handle 3b-i
12//! introduced by **ring group** so a segmenter can exist at all — see
13//! [One writer per ring group, not one writer per `Trunk`](#one-writer-per-ring-group-not-one-writer-per-trunk)
14//! below — per
15//! `docs/superpowers/specs/2026-07-26-media-plane-architecture.md` §1.2.
16//!
17//! This is four bounded rings behind two write handles, split by ring group
18//! (see the section linked just above), and the cursors/queries that read
19//! them: the sample path, the segment log, the event log, and the live-part
20//! log. See
21//! [The event log: 90 kHz absolute, and the B1 crux](#the-event-log-90-khz-absolute-and-the-b1-crux)
22//! below for why the event log needed a third, genuinely different shape —
23//! not just a third copy of `ClassLog`/`SegmentLog` (this module's internal
24//! per-class/per-segment logs) — to resolve the architecture audit's
25//! blocking finding B1.
26//!
27//! # One writer per ring group, not one writer per `Trunk`
28//!
29//! [`Trunk::writer`] used to be the *only* way to publish anything, and its
30//! doc said so in terms broader than the actual reason it exists: "a second
31//! concurrent writer would silently interleave two unrelated publish
32//! sequences into one ring with no way for a reader to tell them apart." That
33//! sentence is true, but it over-generalises from "one ring" to "one
34//! `Trunk`" — and a real consumer wiring this crate up (a segmenter: it reads
35//! samples via a [`SampleCursor`] and, from what it reads, produces segments
36//! and parts) hit the gap that over-generalisation opened: with exactly one
37//! writer for the entire `Trunk`, whichever component takes it for ingest
38//! samples makes it **structurally impossible** for anything else to ever
39//! call [`SegmentWriter::publish_segment`] — no segmenter can exist, and a
40//! segment log/live-part log that can never be filled is not a real feature,
41//! just an unreachable one.
42//!
43//! **The actual invariant, restated correctly**: *within a given ring, there
44//! is exactly one appender* — that is what prevents the interleave the
45//! original sentence worried about, because an interleave requires two
46//! writers racing to append to the *same* ring. It says nothing about a
47//! *different* ring. So the write capability is split by **ring group**,
48//! each still taken at most once via the same `compare_exchange` pattern
49//! [`Trunk::writer`] always used, just with one flag per group instead of
50//! one flag for the whole `Trunk`:
51//!
52//! - [`TrunkWriter`] (via [`Trunk::writer`]) — the **samples + events**
53//! group: [`TrunkWriter::publish`] (the two [`RetentionClass`] sample
54//! rings) and [`TrunkWriter::publish_event`] (the event ring's only
55//! *appending* operation). Held by the ingest driver — the entity that
56//! actually produces both: a demuxed sample, or an inband SCTE-35/`emsg`
57//! event lifted straight off the incoming stream.
58//! - [`SegmentWriter`] (via [`Trunk::segment_writer`]) — the
59//! **segments + parts** group: [`SegmentWriter::publish_segment`],
60//! [`SegmentWriter::publish_part`], plus
61//! [`SegmentWriter::note_segment_start`]/[`SegmentWriter::set_time_anchor`].
62//! Held by whoever owns segmentation.
63//!
64//! The last two methods are grouped here on purpose, not by accident of
65//! naming: neither **appends** an entry to the event ring the way
66//! [`TrunkWriter::publish_event`] does — both *resolve an already-stored*
67//! [`EventAnchor::Segment`]/[`EventAnchor::Utc`] entry **in place** (see
68//! [The event log](#the-event-log-90-khz-absolute-and-the-b1-crux) below).
69//! Since neither is an append, giving them to [`SegmentWriter`] does not
70//! create a second appender for the event ring — [`TrunkWriter::publish_event`]
71//! remains the event ring's only one — while `note_segment_start` in
72//! particular *has* to live wherever segmentation lives: only the segmenter
73//! knows where a segment boundary actually falls (this is the literal B1 fix
74//! — "it cannot be finalised until the segmenter owns a boundary"), so it is
75//! the one entity that can honestly report `note_segment_start`. Placing
76//! `set_time_anchor` alongside it, rather than on [`TrunkWriter`], keeps this
77//! crate's set of "resolve a pending anchor" entry points in one place
78//! instead of splitting a single conceptual capability (anchor resolution)
79//! across two handles for no test or caller that needs it split further; if
80//! a future caller's wall-clock mapping genuinely comes from the ingest side
81//! instead, adding it to [`TrunkWriter`] alongside `publish_event` is
82//! additive, not a breaking re-split of what is here today.
83//!
84//! **Still exactly one writer per group, enforced the same way**: both
85//! [`Trunk::writer`] and [`Trunk::segment_writer`] return `None` on every
86//! call after their first, via their own `AtomicBool` — two concurrent
87//! *sample* writers remain exactly as impossible as before this split; what
88//! changed is that a *segment* writer and a *sample* writer are no longer
89//! forced to be the same handle.
90//!
91//! **The cross-ring ordering question this split raises, answered rather
92//! than left implicit.** A segment is derived from samples the segmenter has
93//! already consumed via its own [`SampleCursor`] — causally, those samples
94//! exist first. With one shared writer, that causal fact was also a
95//! *program-order* fact (one thread called `publish` some number of times,
96//! then called `publish_segment`). With the split, ingest and segmentation
97//! are ordinarily two different threads — could a consumer ever observe
98//! [`SegmentWriter::publish_segment`]'s entry in the segment log *before* the
99//! samples that produced it are visible in the sample ring? **No** — and not
100//! by luck: every ring here (`timed`, `sparse`, `segments`, `events`,
101//! `parts`) still lives inside the *one* `Mutex<TrunkState>` this module has
102//! always used (see [the benchmark verdict](#the-benchmark-verdict-this-design-is-built-around)
103//! below) — splitting the *write handle* did not split the *lock*. The
104//! segmenter can only have samples to build a segment from because its own
105//! `SampleCursor::poll` already returned them, which requires those samples
106//! to have been committed to `state.timed`/`state.sparse` under an *earlier*
107//! acquisition of that same `Mutex`; `SegmentWriter::publish_segment` is then
108//! called afterward, in the segmenter's own program order, under a *later*
109//! acquisition of the identical `Mutex`. Any third party that subsequently
110//! acquires that lock — to poll any ring, from any thread — is guaranteed by
111//! the transitivity of the `Mutex`'s release/acquire ordering to observe at
112//! least everything the segmenter itself had already observed before it
113//! published, samples included. So the specific ordering a consumer must
114//! never see reversed (a segment's constituent samples appearing to lag
115//! behind the segment itself) cannot happen. What *is* true, and is exactly
116//! the existing [`RetentionClass::Timed`]/[`RetentionClass::Sparse`]
117//! precedent extended one layer: the *global* order across unrelated
118//! ring-group activity is not fixed by any one thread's program order any
119//! more — a new, unrelated sample the ingest thread publishes concurrently
120//! may land before or after a segment close the segmenter thread publishes,
121//! in either order, depending on which wins the lock race. Nothing
122//! downstream needs that unrelated cross-ring interleave to be
123//! deterministic (see [Two retention classes](#two-retention-classes-and-why-they-are-two-independent-rings)
124//! below for why this crate already treats "no global cross-ring order" as
125//! an acceptable, load-bearing property, not a defect) — only the *causal*
126//! one, which is what the shared `Mutex` structurally guarantees.
127//!
128//! # Why this module needs `std`, unlike its byte-layer siblings
129//!
130//! [`crate::byte_stage`], [`crate::byte_tap`], and [`crate::byte_merge`] are
131//! `no_std` because each is driven synchronously by a single caller — there is
132//! no cross-thread sharing to arrange. `Trunk` is different in kind: one
133//! writer thread per ring group (ingest for samples/events; a segmenter for
134//! segments/parts, once one exists) and an unbounded set of reader threads
135//! (egress, analysis, DVR) must observe the *same* ring concurrently. That
136//! needs a
137//! shared, lockable interior — `std::sync::{Arc, Mutex}` here, matching
138//! exactly the shape validated by `spikes/trunk-bench` (§3.1 of the spec).
139//! Pulling in a `no_std` spinlock crate just to keep this one module
140//! `no_std`-capable was considered and rejected: every real `Trunk` consumer
141//! (`IngestSession`, `PushEgress`/`SegmentEgress`/`ServedEgress` impls) is
142//! already `std`+`tokio` per the architecture, so there is no `no_std` caller
143//! this would ever serve. This module is therefore `#[cfg(feature = "std")]`
144//! — `media-plane --no-default-features` builds clean without it, exactly
145//! like every other `std`-only corner of the workspace.
146//!
147//! # The benchmark verdict this design is built around
148//!
149//! `spikes/trunk-bench` (commit `acdbf3d0`) measured the naive
150//! one-`Mutex`-guarded-log shape this module implements: **PASS** at the
151//! specced scale (200-track MPTS × 6 readers, 999.97/1000 Mbit/s sustained,
152//! publish mean 5.6 µs / p99 44.3 µs against a ~111 µs budget), but it
153//! **refuted the original O(1)-fan-out premise** — writer cost is **O(N) in
154//! cursor count** (956 ns → 9.98 µs from 1 → 16 readers), because writer and
155//! readers all contend one shared `Mutex`.
156//!
157//! **Consequence, stated where it will actually be read:** see
158//! [`Trunk::subscribe`]. A cursor is for a distinct *consumer of the stream*
159//! — never one per peer of a one-to-many protocol. There is no tee, no
160//! broadcast channel, and no per-consumer queue here, and there will not be
161//! one added to chase higher fan-out: fan-out *is* `subscribe()`, and a
162//! sample's payload is already [`bytes::Bytes`], so handing a clone to a
163//! second, third, or sixteenth reader is a refcount bump, not a copy (see
164//! [Zero-copy fan-out](#zero-copy-fan-out-honestly) below). If a route needs
165//! to serve hundreds or thousands of peers (LL-HLS, WHEP), it takes **one**
166//! cursor here and fans out to its peers itself, at the layer that already
167//! has to hold per-peer state (congestion window, pacing epoch, SRTP
168//! context) anyway.
169//!
170//! The segment log added in this step is a **sibling ring behind the same
171//! one `Mutex`** (the internal `TrunkState`), not a second lock — a
172//! [`SegmentCursor`] contends exactly the lock a [`SampleCursor`] does, so
173//! the same O(N)-in-cursor-count rule and the same single-digit-reader
174//! guidance apply to it verbatim; see [`Trunk::subscribe_segments`] and
175//! [`Trunk::pin_segments`].
176//!
177//! # Two retention classes, and why they are two independent rings
178//!
179//! [`RetentionClass::Timed`] (regular-cadence media) and
180//! [`RetentionClass::Sparse`] (irregular, semantically-critical entries — an
181//! SCTE-35 splice cue, a subtitle sample) are **not** stored in one merged,
182//! globally-ordered log. An earlier design considered exactly that: one
183//! `VecDeque` in strict publish order, with `Sparse` entries migrated to a
184//! small overflow buffer instead of being dropped when the main ring evicted
185//! them. It was rejected as needless complexity for a property nothing
186//! actually needs: nothing downstream reads a `Trunk` expecting a strict
187//! chronological interleave of, say, video samples and SCTE-35 sections —
188//! consumers correlate by PTS/DTS themselves, and the *real* requirement (see
189//! [`RetentionClass::Sparse`]) is only that `Sparse` retention must never be
190//! collateral damage from unrelated `Timed` churn. Two independently
191//! capacity-bounded rings give that guarantee *by construction* — a flood of
192//! video frames cannot evict a still-live splice cue, because there is
193//! nowhere for it to reach it — while a single merged ring would have to
194//! re-implement the same isolation by hand (the rejected overflow-buffer
195//! design above), for no observable benefit. [`SampleCursor::poll`] merges
196//! the two rings only at read time, and documents the (best-effort, not
197//! globally-ordered) precedence it uses.
198//!
199//! # Zero-copy fan-out, honestly
200//!
201//! **This claim was made falsely on this project before**: an earlier
202//! zero-copy fan-out claim was proven only by a test that sliced `Bytes`
203//! itself, while the crate under test contained zero `.slice()` calls of its
204//! own — i.e. the test manufactured the evidence it was supposed to be
205//! checking for. So, stated plainly: **the production path in this module
206//! achieves zero-copy fan-out, not only the test.** the internal per-class
207//! log stores the [`transmux::Sample`] handed to it; [`SampleCursor::poll`] returns it to
208//! a reader via [`Clone::clone`] on the whole `Sample`, which clones
209//! `Sample.data: Bytes` through `Bytes`'s own `Clone` impl — an `Arc`-style
210//! refcount bump, not a byte copy. There is no `.slice()`, no
211//! `Bytes::copy_from_slice`, and no re-allocation anywhere on this path. The
212//! test in this module (`payload_is_shared_not_copied_across_cursors`)
213//! asserts `Bytes::as_ptr()` *identity* across multiple cursors reading the
214//! same published entry, precisely so it cannot be satisfied by two payloads
215//! that merely have equal contents — and a mutation swapping the `clone()`
216//! for a real copy is recorded as run against it (see that test's doc
217//! comment).
218//!
219//! The segment log added in this step makes the **same** claim, honestly, on
220//! the **same** terms: [`SegmentEntry::bytes`] is [`bytes::Bytes`],
221//! [`SegmentCursor::poll`] hands it back via `Clone` on the whole
222//! [`SegmentEntry`] (which clones `Bytes` through `Bytes`'s own `Clone`), and
223//! there is no `.slice()`/`copy_from_slice`/re-allocation anywhere on that
224//! path either. `segment_bytes_are_shared_not_copied_across_cursors` asserts
225//! the same pointer-identity property for segments that
226//! `payload_is_shared_not_copied_across_cursors` asserts for samples — this
227//! is the **production** path achieving zero-copy fan-out, not a test
228//! manufacturing its own evidence.
229//!
230//! # The DVR contradiction: losslessness from retention, not back-pressure
231//!
232//! A DVR/archive consumer must not miss a segment — a hole in a recording is
233//! a defect, not a degradation, unlike a dropped video frame. But the writer
234//! must **never** block, for exactly the reason stated everywhere else in
235//! this module: a stalled archive writer must not stall live ingest. Those
236//! two requirements contradict each other directly if "losslessness" is
237//! implemented the obvious way — by making the writer wait for a slow
238//! archive reader.
239//!
240//! **The resolution: losslessness comes from retention, not from
241//! back-pressure.** A [`SegmentCursor`] obtained via [`Trunk::pin_segments`]
242//! *pins* every segment it has not yet consumed — the log will not evict a
243//! pinned entry as a matter of course, the way it freely evicts for an
244//! ordinary [`Trunk::subscribe_segments`] cursor. "Consumed" here means
245//! "returned by [`SegmentCursor::poll`]" — the same progress counter that
246//! already governs in-order delivery does double duty as the pin floor,
247//! rather than adding a second, explicit acknowledge-after-durable-write API
248//! call. That two-call shape (poll to receive, then a separate `ack` once
249//! the archive write actually lands on disk) was considered — it is the more
250//! conservative choice, since a consumer that has polled a segment but not
251//! yet finished writing it to disk is not truly safe from loss if the trunk
252//! evicts under it — and rejected for *this* step: it doubles the API
253//! surface and the bookkeeping (two offsets per pin instead of one) for a
254//! distinction (poll's delivery vs. a durable write landing) this step has
255//! no test that needs, since nothing downstream is implemented yet
256//! (`docs/superpowers/plans/2026-07-26-media-plane-implementation.md` step
257//! 3d's `SegmentEgress`/DVR writer is what would consume it). If that step
258//! needs the finer-grained split, it is additive — a second, later
259//! acknowledgement point on the same pin — not a breaking change to this
260//! one.
261//!
262//! **Pinning is bounded, and by design there is no second capacity knob for
263//! it**: a pin is measured against exactly [`TrunkConfig::segment_capacity`],
264//! the same bound that governs ordinary eviction for every cursor. There is
265//! no independent "how far behind may a pin fall" setting to tune
266//! separately and get wrong. When the segment log is at capacity and the
267//! next [`SegmentWriter::publish_segment`] would evict an entry some pin has
268//! not yet consumed, the bound has been hit, and something genuinely has to
269//! give — the caller decided what, in advance, via the [`ArchiveOverrun`]
270//! passed to [`Trunk::pin_segments`]:
271//!
272//! - [`ArchiveOverrun::Gap`] (**the default**) — evict the pinned entry
273//! anyway, and tell that cursor it lost data
274//! ([`SegmentCursorItem::Gap`]). The recording gets a hole; the live
275//! stream and every other cursor are unaffected.
276//! - [`ArchiveOverrun::StallIngest`] — apply real back-pressure:
277//! [`SegmentWriter::publish_segment`] blocks until this cursor consumes
278//! enough to release its pin (or is dropped). **The only place in this
279//! entire design where a reader may block the writer** — opt-in,
280//! documented loudly here and on the variant itself, and never the
281//! default.
282//! - [`ArchiveOverrun::Terminate`] — drop the cursor's pin outright instead
283//! of gapping the recording or stalling ingest; the cursor is done
284//! ([`SegmentCursorItem::Terminated`]) and the log continues without it.
285//!
286//! This is a genuine three-way trade between the recording, the live
287//! stream, and the archive consumer — **no option is free**, and there is
288//! deliberately no fourth "just make it work" variant: any such variant
289//! would have to secretly pick one of the three trade-offs above anyway
290//! (drop bytes, block the writer, or drop the consumer), just without
291//! naming which — which is worse, not better.
292//!
293//! # The event log: 90 kHz absolute, and the B1 crux
294//!
295//! `TrunkState` (the shared state behind a `Trunk`) holds the two per-class
296//! sample logs, the segment log, and now the event log — a **sibling ring
297//! behind the same one `Mutex`**, exactly the pattern the segment log
298//! established (`TrunkState::events: EventLog`, `Trunk::subscribe_events`/
299//! `events_between`/`events_in_segment` shaped like `Trunk::subscribe`/
300//! `Trunk::subscribe_segments`, `TrunkConfig::event_capacity` alongside
301//! `timed_capacity`/`sparse_capacity`/`segment_capacity`). Where it is
302//! genuinely a new shape, not a third `ClassLog`/`SegmentLog` copy, is
303//! its *clock* and its *addressing* — both forced by architecture audit
304//! finding B1
305//! (`docs/superpowers/specs/2026-07-26-media-plane-architecture.md` §0/§1.2).
306//!
307//! **What B1 got wrong.** Revision 1 of the spec claimed one time model for
308//! everything the plane carries: an absolute `i64` in the *producing
309//! track's* timescale. That is false in two ways this project already
310//! parses, and both are events, not samples:
311//!
312//! - `splice_schedule.utc_splice_time` (SCTE-35 §9.7.4) is **GPS-epoch
313//! UTC** — not a media timestamp in any track's timescale at all.
314//! - `emsg` version 0's `presentation_time_delta` (ISO/IEC 23009-1
315//! §5.10.3.3) is **segment-relative** — its value only means something
316//! once you know which segment it lands in, and that segment's earliest
317//! presentation time is not knowable until the segmenter has actually cut
318//! the boundary. `timed_metadata::convert::emsg_convert` already encodes
319//! this exact arithmetic (`T = EPT + presentation_time_delta`) for
320//! *converting* one emsg to another; the event log's job is different —
321//! it has to hold the delta *honestly unresolved* for however long the
322//! boundary is unknown, which a stateless conversion function has no
323//! reason to model.
324//!
325//! Neither of those is expressible as a single struct field without either
326//! (a) losing information (which timescale? relative to what?) or (b)
327//! **fabricating** a resolution that has not actually happened yet — an
328//! event log that stores a plausible-looking media time for a
329//! `splice_schedule` cue before any wall-clock↔media-clock mapping exists,
330//! or for an `emsg` v0 before its segment's start is known, has invented
331//! data. **The failure mode is not a crash: it is an ad break firing at the
332//! wrong wall-clock instant**, because a plausible-but-wrong media time is
333//! indistinguishable from a correct one until playout.
334//!
335//! **Why 90 kHz absolute, not per-track timescale.** A single `Media` can
336//! carry several tracks at several timescales (48 kHz audio, a 25 fps
337//! video track at 90 000, a subtitle track with none at all) — there is no
338//! one track whose timescale the *event* log could borrow without an
339//! arbitrary, undocumented choice among them. [`EventAnchor::Media`]
340//! therefore carries [`timed_metadata::MediaTime`] — 90 kHz ticks,
341//! wrap-unrolled, the same clock SCTE-35's own `pts_time` already uses —
342//! rather than any one track's clock. This is also why the event log is a
343//! genuinely separate ring from the sample rings, not a third
344//! [`RetentionClass`]: a [`transmux::Sample`] is timestamped in its
345//! *track's* clock ([`transmux::Sample::pts`]/`dts`, per §4 of the spec);
346//! an event lives on the trunk's own, track-independent clock.
347//!
348//! **Carries [`timed_metadata::TimedEvent`], not a parallel type.** It is
349//! owned, lossless, `#[non_exhaustive]`, and already published (0.4.0, live
350//! on crates.io) — [`EventEntry::event`] stores it verbatim rather than
351//! re-deriving a second event representation this crate would then have to
352//! keep in sync by hand. `mp4_emsg::EmsgBox<'a>` is *borrowed* and cannot
353//! outlive the buffer it was parsed from, so it cannot sit in a `'static`
354//! ring; [`timed_metadata::SourcePayload::Emsg`] is already its owned form
355//! (scheme/value/verbatim `message_data`), and is what ends up inside the
356//! stored `TimedEvent` for an `emsg`-sourced entry.
357//!
358//! **The B1 crux: [`EventAnchor`] — an unresolved event stays honestly
359//! unresolved.** Every entry's addressability is one of three states, and
360//! there is deliberately no path from `Segment`/`Utc` to `Media` other than
361//! the specific fact each one is waiting for actually arriving:
362//!
363//! - [`EventAnchor::Media`] — already on the trunk's 90 kHz clock (a
364//! `splice_time` PTS post-wrap-unroll, or an already-absolute `emsg` v1).
365//! - [`EventAnchor::Segment`] — an `emsg` v0's `presentation_time_delta`
366//! plus the `segment_number` it is relative to. Stays exactly this
367//! variant — addressable by segment number, **not** by media time —
368//! until [`SegmentWriter::note_segment_start`] reports that segment's
369//! start, at which point this module's internal event log resolves it
370//! **in place**, computed from *that segment's own* reported start —
371//! never "whichever segment happens to be currently open", which would
372//! silently produce *a* segment instead of *the* segment the emsg
373//! actually named.
374//! - [`EventAnchor::Utc`] — a GPS/UTC instant (`splice_schedule`) with no
375//! media-timeline position at all. Stays exactly this variant — not
376//! returned by [`Trunk::events_between`] or [`Trunk::events_in_segment`],
377//! because there is no honest media time to filter on — until
378//! [`SegmentWriter::set_time_anchor`] gives the event log a
379//! [`timed_metadata::TimeAnchor`] to translate through. This is the
380//! literal B1 test: an event with only a wall-clock time and no anchor
381//! must never be handed a fabricated media time.
382//!
383//! `epoch_ms_to_media` (the UTC→media direction) is the mirror image of
384//! [`timed_metadata::TimeAnchor::media_to_epoch_ms`] (which only goes the
385//! other way) — plain affine algebra, **not** a reimplementation of
386//! [`timed_metadata::Timeline`]'s 33-bit wrap-unroll, which this module
387//! reuses rather than hand-rolls: every `MediaTime` this ring ever stores
388//! either came out of `Timeline::push_scte35` already unrolled, or is
389//! computed from one that did (`Segment`/`Utc` resolution only ever adds a
390//! non-negative delta or an anchor-relative offset to an already-unrolled
391//! value).
392//!
393//! **Dual addressing: media time *and* segment, both, not either** — because
394//! a manifest renderer needs "the events in segment N" while a playback
395//! scheduler needs "the events between T1 and T2", and neither is a special
396//! case of the other. [`Trunk::events_between`] answers the first
397//! (half-open `[from, to)` over every currently-`Media`-resolved entry);
398//! [`Trunk::events_in_segment`] answers the second, by consulting
399//! `EventLog::segment_starts` — a small boundary table, populated by
400//! [`SegmentWriter::note_segment_start`], bounded by the **same**
401//! `TrunkConfig::event_capacity` rather than a second, independent knob
402//! (exactly [`TrunkConfig::segment_capacity`]'s "no second capacity knob"
403//! precedent for pinning). Both queries only ever return `Media`-resolved
404//! entries — an entry still `Segment`/`Utc`-anchored is not fabricated a
405//! position just to satisfy either query.
406//!
407//! Both point-in-time queries read the same log a subscribed
408//! [`EventCursor`] does (via [`Trunk::subscribe_events`]) — the same
409//! single-`Mutex`, single-digit-reader-by-design, in-band-loss-reporting,
410//! writer-never-blocks shape [`Trunk::subscribe`]/[`Trunk::subscribe_segments`]
411//! already established, reused verbatim rather than reconsidered: an
412//! `EventCursor` sees an entry (and a `Lagged` loss report, if it fell
413//! behind [`TrunkConfig::event_capacity`]'s eviction) the moment it is
414//! published, whether or not it has resolved yet, while the two query
415//! methods are a snapshot of what has resolved *so far*.
416//!
417//! `SegmentEgress` and tiered `Retention` (plan steps 3d/3e — an egress
418//! trait that owns one [`SegmentCursor`] and pushes to DVR/MABR/ROUTE/Smooth,
419//! and a hot/cold archive store behind it) are **not** built here, and their
420//! attachment point is exactly [`Trunk::pin_segments`]: a `SegmentEgress`
421//! implementation is the caller this step's [`ArchiveOverrun`] was written
422//! for — it takes a pinning cursor with whichever policy its durability
423//! contract requires (`StallIngest` for "this archive must never have a
424//! hole", `Gap` for "best-effort is fine"), drains [`SegmentCursor::poll`],
425//! and writes [`SegmentEntry::bytes`] to its store. Nothing in this step's
426//! shape needs to change to make room for that; it is exactly the sample
427//! path's `PushEgress`-owns-one-`SampleCursor` story repeated one layer up.
428//! (`SegmentEgress`/`Retention` are named here only to document the
429//! attachment point per this step's brief — neither type exists in this
430//! crate yet.)
431//!
432//! # The live-part log: parts before their segment closes
433//!
434//! Step 3d built `ServedEgress`/`EgressResponse::Await` and, per its own
435//! brief, read `hls-runtime/src/server/` before finishing to report what
436//! did **not** fit. It found the segment log alone cannot serve LL-HLS at
437//! all: RFC 8216bis's entire low-latency mechanism is **part-level**
438//! availability ("does part 3 of the segment currently being written
439//! exist"), and before this step there was nowhere in this `Trunk` to ask
440//! that — the segment log holds only *finished* segments. This step adds a
441//! fourth ring, the live-part log (`TrunkState::parts: PartLog`), storing
442//! [`PartEntry`] exactly the way this module's internal `SegmentLog` stores [`SegmentEntry`] —
443//! same evict-then-push shape, same zero-copy-fan-out claim (see
444//! [Zero-copy fan-out](self#zero-copy-fan-out-honestly)) — bounded by the
445//! new [`TrunkConfig::part_capacity`].
446//!
447//! **Addressed the way a client actually asks**: not a moving cursor
448//! position, but a direct `(segment_number, part_index)` key —
449//! [`Trunk::part_bytes`] and [`Trunk::parts_in_segment`], the live-part
450//! counterparts of [`Trunk::events_between`]/[`Trunk::events_in_segment`].
451//! This is deliberate, not an oversight of "should there also be a
452//! `PartCursor`": a `ServedEgress` implementing LL-HLS resolves *random*
453//! requests ("is part 3.2 ready") against whatever is currently true, not a
454//! sequential stream of every part ever produced — exactly the same
455//! resolve-a-request-against-shared-state shape
456//! [`crate::egress::ServedEgress::resolve`]'s own module doc already argues
457//! for the event log's snapshot queries. No `PartCursor` is added because no
458//! test in this step (or in `crate::egress`) needs one; streaming every part
459//! as it is produced (a hypothetical future low-latency `PushEgress`) is
460//! additive later, not a gap today.
461//!
462//! **What happens when the parent segment closes — decided, not left
463//! implicit**: [`SegmentWriter::publish_segment`] does **not** touch the
464//! live-part log at all. A part stays addressable via [`Trunk::part_bytes`]
465//! for exactly as long as [`TrunkConfig::part_capacity`]'s ordinary
466//! evict-oldest bound has not yet reclaimed it — whether its parent segment
467//! is still open or has already closed makes no difference to this ring.
468//! Three alternatives were considered and rejected:
469//!
470//! - *Roll a closed segment's parts into its [`SegmentEntry`]* — rejected:
471//! `SegmentEntry::bytes` is already the whole muxed segment; attaching its
472//! parts too would store the same encoded media twice (once whole, once
473//! split), the opposite of this crate's zero-copy-fan-out discipline, for
474//! a property ([`Trunk::part_bytes`] already answers "is this part ready")
475//! nothing needs.
476//! - *Evict a segment's parts the instant it closes* — rejected: this is
477//! the exact bug `hls_runtime::server::MediaStore`'s own `recent_parts`
478//! buffer exists to prevent (documented there as "the segmenter emits a
479//! segment's final part and closes the segment in the same pipeline
480//! step... without this the part is evicted microseconds after it
481//! appears — before the blocked part request can wake"). A `ServedEgress`
482//! built on this `Trunk` needs the same guarantee, and immediate eviction
483//! on close would remove it.
484//! - *A second, shorter-lived "recently closed" bound, chained after the
485//! live bound* — this is what `MediaStore` actually does
486//! (`live_parts` + a separately-capped `recent_parts`, doubling worst-case
487//! retention) — rejected here as the "second knob" this file's precedent
488//! argues against: one bound, applied uniformly regardless of open/closed
489//! status, gives the same client-visible guarantee (a just-closed part
490//! stays fetchable) without a second, independently-tunable lifetime that
491//! can disagree with the first.
492//!
493//! **What a client requesting a just-rolled part receives**: the same
494//! answer as the instant before the segment closed — `Some(bytes)` from
495//! [`Trunk::part_bytes`] — because closing did not touch this ring. It
496//! becomes `None` only once ordinary `part_capacity` eviction reclaims it,
497//! at which point this ring cannot distinguish "evicted" from "never
498//! existed"; a `ServedEgress` wanting RFC 8216bis's sharper "will never
499//! exist, stop waiting" signal for a part of an *already-closed* segment
500//! (`hls_runtime::server::MediaStore::resolve_resource`'s
501//! `ResourceOutcome::NotFound` case) gets that distinction the same way
502//! `MediaStore` itself does: by also consulting [`Trunk::last_closed_segment`]
503//! — if the requested part's `segment_number` is at or before that value
504//! and [`Trunk::part_bytes`] answers `None`, the part will never arrive; if
505//! it is beyond it, the segment (and the part) may still be produced.
506//!
507//! # The reader-wake primitive: `listen`, not one registration per remote peer
508//!
509//! The second gap step 3d found, recorded rather than solved: every `Trunk`
510//! reader was a synchronous, non-blocking `poll()`, so a `ServedEgress`
511//! implementing RFC 8216bis §6.2.5.2 blocking reload had nothing to wait
512//! *on* — only a poll-with-backoff loop. [`Trunk::listen`] closes that gap
513//! by handing back a [`ProgressListener`] wrapping
514//! [`event_listener::EventListener`] — the exact runtime-agnostic primitive
515//! `hls_runtime::server::MediaStore::listen` already returns (an already
516//! std+`event-listener`-feature dependency of this crate's sibling, and now
517//! of this one), not a hand-rolled parallel mechanism, so a caller ports
518//! mechanically: `.await` it under any executor, or call
519//! [`ProgressListener::wait_deadline`] with no executor at all — precisely
520//! `MediaStore::listen`'s own two documented ways to wait.
521//!
522//! **The writer never blocks on this.** [`SegmentWriter::publish_part`]/
523//! [`SegmentWriter::publish_segment`] call `Event::notify(usize::MAX)`, which
524//! wakes every currently-registered listener without waiting for any of
525//! them to actually resume running — the same non-blocking-producer
526//! guarantee this module makes everywhere else
527//! ([`TrunkWriter::publish`]'s doc), extended to a wake channel instead of a
528//! data ring. A registered [`ProgressListener`] that nobody ever polls or
529//! waits on again (a vanished HTTP peer, a wedged executor) costs the
530//! writer nothing beyond that one `notify` call's O(waiter-count) fan-out —
531//! it never becomes a wait.
532//!
533//! **Bounded, and reusing [`TrunkConfig::part_capacity`] rather than a sixth
534//! knob.** [`Trunk::listen`] refuses (`None`) once `part_capacity`
535//! concurrent [`ProgressListener`]s are outstanding. This is deliberately
536//! **not** sized "one registration per remote viewer": that would repeat
537//! exactly the O(N)-in-cursor-count mistake [`Trunk::subscribe`]'s own docs
538//! warn against for data cursors, now for wake registrations instead of
539//! poll positions. The intended shape mirrors `subscribe`'s "one cursor per
540//! distinct consumer, never one per peer" rule: a `ServedEgress` adapter
541//! serving a thousand LL-HLS viewers takes **one** (or a small, fixed
542//! number of) [`Trunk::listen`] registration(s) for the route and fans the
543//! single wake-up out to its own thousand blocked HTTP handlers itself,
544//! using its own broadcast mechanism — exactly the same layering
545//! [`crate::egress::PushEgress`] already requires for sample fan-out. Under
546//! that shape, `part_capacity`-many concurrent *distinct-consumer*
547//! registrations is generous headroom, not a production ceiling; if a
548//! caller instead wires one HTTP request directly to one `Trunk::listen`
549//! call each (mirroring how `MediaStore`'s single, uncapped `Event` is used
550//! today), the cap is exactly the backstop this step exists to add — a
551//! caller hitting `None` must treat it the same way it treats an already-
552//! expired [`crate::egress::AwaitPolicy`]: answer the request as
553//! unavailable now rather than waiting with no slot to wait in. Composing
554//! with [`crate::egress::AwaitPolicy`]'s deadline is the caller's
555//! conversion of `AwaitPolicy::deadline` (a [`Timestamp`]) to the
556//! `std::time::Instant` it already anchors that `Timestamp` to, passed to
557//! [`ProgressListener::wait_deadline`] (or wrapped in the caller's own
558//! executor timeout around the `Future` impl) — see
559//! [`ProgressListener::wait_deadline`]'s own doc.
560
561use std::collections::{HashMap, VecDeque};
562use std::future::Future;
563use std::num::NonZeroUsize;
564use std::pin::Pin;
565use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
566use std::sync::{Arc, Condvar, Mutex};
567use std::task::{Context, Poll};
568use std::time::Duration;
569
570use broadcast_common::stage::Timestamp;
571use bytes::Bytes;
572use event_listener::{Event, EventListener, Listener};
573use timed_metadata::{MediaTime, PTS_HZ, TimeAnchor, TimedEvent};
574use transmux::{Sample, SegmentMeta, TrackSpec};
575
576/// Which retention discipline a published entry follows once inside the
577/// [`Trunk`]'s sample ring.
578///
579/// Named `RetentionClass`, not `Retention` — plan step 3e's tiered hot/cold
580/// archive policy (`docs/superpowers/plans/2026-07-26-media-plane-implementation.md`)
581/// owns the name `Retention` for an unrelated, later concept. This is the
582/// orthogonal, in-ring question of "how eagerly can this entry be evicted",
583/// decided per [`TrunkWriter::publish`] call by whoever is feeding the
584/// writer — it reflects a *track's* nature (video/audio vs. an SCTE-35
585/// section PID), not something intrinsic to a [`transmux::Sample`] itself,
586/// so it is not a field the spec's `Sample` type carries.
587#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
588#[non_exhaustive]
589pub enum RetentionClass {
590 /// Regular-cadence media samples (audio, video, ...): count-bounded, and
591 /// ordinary eviction is reported to a lagging [`SampleCursor`] as
592 /// [`SampleCursorItem::Lagged`] — a consumer that misses a video frame is
593 /// gapped, not wrong; it resumes from the next sample.
594 Timed,
595 /// Irregular, semantically-critical entries — an SCTE-35 splice cue, a
596 /// subtitle sample — where losing one leaves a consumer's *derived
597 /// state* wrong, not merely gapped: a missed splice cue means splicing
598 /// in the wrong place, or not at all.
599 ///
600 /// # The retention rule
601 ///
602 /// A `Sparse` entry lives in a ring bounded **independently** of the
603 /// `Timed` ring ([`TrunkConfig::sparse_capacity`], separate from
604 /// [`TrunkConfig::timed_capacity`]). It is therefore never evicted
605 /// "merely because a time window rolled" on the unrelated `Timed`
606 /// class: no volume of video/audio publishes can push a still-live
607 /// splice cue out of the trunk, because `Timed` publishes never touch
608 /// the `Sparse` ring at all. A `Sparse` entry is only evicted once
609 /// `Sparse` publish volume *itself* exceeds the `Sparse` ring's own
610 /// bound — and when that happens, [`SampleCursor::poll`] reports it as
611 /// [`SampleCursorItem::Degraded`], not ordinary `Lagged`: a distinct,
612 /// stronger signal, because the consumer's semantic state (e.g. "where
613 /// the next ad break splices") is now wrong. A consumer that sees
614 /// `Lagged` should simply resume from the next sample; a consumer that
615 /// sees `Degraded` should treat its derived state as unsynchronised
616 /// until the next authoritative signal (a fresh cue, a manifest
617 /// reload) re-establishes it — resuming silently would splice on stale
618 /// information.
619 Sparse,
620}
621
622/// Construction parameters for a [`Trunk`].
623///
624/// # Why every capacity is a [`NonZeroUsize`], not a validated `usize`
625///
626/// A zero capacity is not a value this type rejects — it is a value this type
627/// **cannot represent**. Every ring in this module evicts its oldest entry
628/// when `entries.len() == capacity`, so a zero capacity would evict every
629/// entry the instant it was pushed, and a zero waiter cap would make
630/// [`Trunk::listen`] incapable of ever registering anybody: not a
631/// configuration, a broken one.
632///
633/// Two weaker designs were considered and rejected:
634///
635/// - **Panicking on zero in [`Trunk::new`]** (what this type did before):
636/// internally consistent, but a library that panics on a value which
637/// arrives *from a file* is a real operational hazard, not a style
638/// question — `multimux` takes its routes from a JSON config, so once
639/// these capacities become operator-configurable a stray `0` would take
640/// down the server process instead of producing a config error. It also
641/// contradicted `transmux::ProgressiveDemux::new`'s deliberate
642/// panic-to-fallible change, which is exactly the kind of
643/// two-crates-apart inconsistency that makes an API feel arbitrary.
644/// - **A fallible `TrunkConfig::new -> Result<Self, _>`** (the
645/// `ProgressiveDemux` shape): correct, but strictly worse *here*.
646/// `ProgressiveDemux` already returns `Result` as part of its `Stage`
647/// contract and already has an `Error` type; `TrunkConfig` has neither, so
648/// this would mean inventing a construction error type and threading
649/// `?`/`unwrap` through every construction site to encode one bit of
650/// information the type system can carry for free. `NonZeroUsize` puts
651/// the invariant *in the signature*, where a reader learns it without
652/// reading this doc — and, for the JSON-config hazard specifically, a
653/// `serde` deserialize of `0` into a `NonZeroUsize` field already fails as
654/// an ordinary deserialization error at the config boundary, with no
655/// hand-written check and no panic.
656#[derive(Debug, Clone, Copy)]
657#[non_exhaustive]
658pub struct TrunkConfig {
659 /// Bound, in entry count, on the [`RetentionClass::Timed`] ring.
660 pub timed_capacity: NonZeroUsize,
661 /// Bound, in entry count, on the [`RetentionClass::Sparse`] ring —
662 /// independent of `timed_capacity`; see [`RetentionClass::Sparse`] for
663 /// why that independence is the entire point of the retention rule.
664 pub sparse_capacity: NonZeroUsize,
665 /// Bound, in entry count, on the segment log. **Also** the bound a
666 /// pinning [`SegmentCursor`]'s retention is measured against — see
667 /// [The DVR contradiction](self#the-dvr-contradiction-losslessness-from-retention-not-back-pressure)
668 /// for why there is deliberately no second, independent "pin depth"
669 /// knob.
670 pub segment_capacity: NonZeroUsize,
671 /// Bound, in entry count, on the event log — **and** on its segment
672 /// boundary table (`EventLog::segment_starts`). See
673 /// [The event log](self#the-event-log-90-khz-absolute-and-the-b1-crux)
674 /// for why a segment-relative event's target boundary shares this one
675 /// knob rather than getting a second, independently-tuned one —
676 /// exactly [`TrunkConfig::segment_capacity`]'s "no second capacity
677 /// knob" precedent for pinning.
678 pub event_capacity: NonZeroUsize,
679 /// Bound, in entry count, on the live-part log (step 3b-iv) — **and**
680 /// on how many concurrent [`Trunk::listen`] registrations this trunk
681 /// will honor at once. See
682 /// [The live-part log](self#the-live-part-log-parts-before-their-segment-closes)
683 /// for why a part-of-the-open-segment shares this one knob for both
684 /// jobs, rather than getting a second, independently-tuned "how many
685 /// waiters" setting — the third instance of this file's "no second
686 /// capacity knob" precedent (after [`TrunkConfig::segment_capacity`]'s
687 /// pin reuse and [`TrunkConfig::event_capacity`]'s `segment_starts`
688 /// reuse).
689 pub part_capacity: NonZeroUsize,
690}
691
692impl TrunkConfig {
693 /// Build a config with all five ring capacities. Nothing is validated
694 /// here, and nothing needs to be: [`NonZeroUsize`] makes the only
695 /// invalid value unrepresentable rather than merely rejected — see
696 /// [this type's own docs](TrunkConfig#why-every-capacity-is-a-nonzerousize-not-a-validated-usize)
697 /// for why that beats both the panic this replaced and a fallible
698 /// constructor.
699 pub fn new(
700 timed_capacity: NonZeroUsize,
701 sparse_capacity: NonZeroUsize,
702 segment_capacity: NonZeroUsize,
703 event_capacity: NonZeroUsize,
704 part_capacity: NonZeroUsize,
705 ) -> Self {
706 TrunkConfig {
707 timed_capacity,
708 sparse_capacity,
709 segment_capacity,
710 event_capacity,
711 part_capacity,
712 }
713 }
714}
715
716/// One retention class's bounded, append-ordered log of `(track_id, Sample)`
717/// entries.
718///
719/// Bench-identical bounding: when full, the oldest entry is evicted and
720/// `base` (the count of entries ever evicted from *this* log) advances by
721/// one; `published` is the count of entries ever pushed. A cursor's lag for
722/// this class is computed purely from `base` vs. how much of it the cursor
723/// has consumed — see [`SampleCursor::poll`].
724struct ClassLog {
725 entries: VecDeque<(u32, Sample)>,
726 base: u64,
727 published: u64,
728 capacity: usize,
729}
730
731impl ClassLog {
732 fn new(capacity: usize) -> Self {
733 ClassLog {
734 entries: VecDeque::with_capacity(capacity),
735 base: 0,
736 published: 0,
737 capacity,
738 }
739 }
740
741 /// Push one entry, evicting the oldest if the log is already at
742 /// `capacity`. Never rejects, never blocks — this is what lets
743 /// [`TrunkWriter::publish`] complete unconditionally regardless of how
744 /// far behind any reader has fallen.
745 fn push(&mut self, track_id: u32, sample: Sample) {
746 if self.entries.len() == self.capacity {
747 self.entries.pop_front();
748 self.base += 1;
749 }
750 self.entries.push_back((track_id, sample));
751 self.published += 1;
752 }
753}
754
755/// One finished media segment recorded by the segment log, in playlist
756/// order.
757///
758/// Reuses [`transmux::SegmentMeta`] for exactly what it already models — the
759/// per-segment discontinuity bit [`transmux::Segmenter::take_ready_with_meta`]
760/// returns — by holding the whole type rather than copying its one field out
761/// into a `discontinuous: bool` of this struct's own; a field `SegmentMeta`
762/// gains later is picked up here for free. It does **not** fit whole,
763/// though, and this struct says so rather than pretending it does: nothing
764/// in `transmux` computes a segment's wall-clock duration, its `moof`/`mfhd`
765/// sequence number, or its position on *this trunk's* absolute timeline —
766/// those are properties of the log a segment lands in, not of the segmenter
767/// that produced its bytes, so they are new fields here, supplied by
768/// whoever is feeding [`SegmentWriter::publish_segment`], exactly as
769/// `track_id`/[`RetentionClass`] are supplied by whoever feeds
770/// [`TrunkWriter::publish`].
771#[derive(Debug, Clone)]
772#[non_exhaustive]
773pub struct SegmentEntry {
774 /// The segment's encoded bytes. `Bytes`, not `Vec<u8>`, for the same
775 /// reason as [`transmux::Sample::data`]: fan-out to every
776 /// [`SegmentCursor`] reading this entry is a refcount bump, not a copy —
777 /// see [Zero-copy fan-out](self#zero-copy-fan-out-honestly).
778 pub bytes: Bytes,
779 /// This segment's `moof`/`mfhd` sequence number (1-based, matching
780 /// [`transmux::Segmenter`]'s own numbering) — what a consumer needs to
781 /// name the segment in a playlist or manifest.
782 pub sequence_number: u32,
783 /// This segment's duration, wall-clock — what a consumer needs for
784 /// `#EXTINF`/`<S d="...">`.
785 pub duration: Duration,
786 /// This segment's start position on the trunk's absolute timeline.
787 pub timeline_position: Timestamp,
788 /// The discontinuity bit from the segmenter itself; see this struct's
789 /// own doc for why it is reused by embedding the whole type, not
790 /// re-derived as a field of this struct.
791 pub meta: SegmentMeta,
792}
793
794impl SegmentEntry {
795 /// Build one segment log entry.
796 pub fn new(
797 bytes: impl Into<Bytes>,
798 sequence_number: u32,
799 duration: Duration,
800 timeline_position: Timestamp,
801 meta: SegmentMeta,
802 ) -> Self {
803 SegmentEntry {
804 bytes: bytes.into(),
805 sequence_number,
806 duration,
807 timeline_position,
808 meta,
809 }
810 }
811}
812
813/// The caller-chosen policy for what happens when a **pinning**
814/// [`SegmentCursor`] (from [`Trunk::pin_segments`]) has not yet consumed an
815/// entry the segment log needs to evict because it is at
816/// [`TrunkConfig::segment_capacity`].
817///
818/// See [The DVR contradiction](self#the-dvr-contradiction-losslessness-from-retention-not-back-pressure)
819/// for why this is a three-way, caller-chosen trade with no free option and
820/// deliberately no fourth "just make it work" variant.
821#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
822#[non_exhaustive]
823pub enum ArchiveOverrun {
824 /// Evict the pinned entry anyway, and report the loss to this cursor as
825 /// [`SegmentCursorItem::Gap`] on its next [`SegmentCursor::poll`]. The
826 /// recording gets a hole; live ingest and every other cursor are
827 /// unaffected. **The default** — a pinning cursor that does not choose
828 /// otherwise gets availability over completeness, the same trade
829 /// [`RetentionClass::Timed`]'s ordinary `Lagged` already makes for the
830 /// sample ring.
831 Gap,
832 /// Apply real back-pressure: [`SegmentWriter::publish_segment`] blocks
833 /// until this cursor consumes far enough to release its pin (or the
834 /// cursor is dropped). **The only place in this entire design where a
835 /// reader may block the writer** — opt-in only, never the default;
836 /// choosing it means a wedged or malicious archive consumer can stall
837 /// segment publication indefinitely.
838 StallIngest,
839 /// Drop this cursor's pin outright instead of gapping the recording or
840 /// stalling ingest: the cursor is terminated (its next `poll` returns
841 /// [`SegmentCursorItem::Terminated`], and every `poll` after that
842 /// returns `None`) and the log continues without it.
843 Terminate,
844}
845
846impl Default for ArchiveOverrun {
847 /// [`ArchiveOverrun::Gap`] — see that variant's doc for why gapping the
848 /// recording, rather than stalling ingest, is the safe default.
849 fn default() -> Self {
850 ArchiveOverrun::Gap
851 }
852}
853
854/// Per-pinning-cursor bookkeeping the segment log consults, at each
855/// [`SegmentWriter::publish_segment`], to decide whether evicting the oldest
856/// entry is safe.
857struct PinState {
858 /// This pin's own read progress: the same role [`SampleCursor`]'s local
859 /// `*_consumed` fields play, made visible to the *writer* instead of
860 /// staying purely cursor-local, because eviction has to consult it
861 /// *before* evicting, not merely report loss after the fact.
862 /// "Acknowledged" (module docs) means "returned by
863 /// [`SegmentCursor::poll`]" — see the module docs' DVR section for why a
864 /// separate ack-after-durable-write step was considered and rejected
865 /// for this step.
866 consumed: u64,
867 /// The policy chosen at [`Trunk::pin_segments`] time.
868 policy: ArchiveOverrun,
869 /// Set once [`ArchiveOverrun::Terminate`] has fired for this pin; the
870 /// next `poll` on the owning cursor reports
871 /// [`SegmentCursorItem::Terminated`] and removes this entry.
872 terminated: bool,
873}
874
875/// The segment log: a bounded, append-ordered log of [`SegmentEntry`]
876/// values, plus the pin bookkeeping [`ArchiveOverrun`] needs.
877///
878/// Evict-then-push shape identical to [`ClassLog`] — `base`/`published`
879/// mean exactly the same thing here as there — with one addition: a publish
880/// that would evict an entry a pinning cursor has not yet consumed does not
881/// evict unconditionally; [`SegmentWriter::publish_segment`] consults that
882/// pin's [`ArchiveOverrun`] first.
883struct SegmentLog {
884 entries: VecDeque<SegmentEntry>,
885 base: u64,
886 published: u64,
887 capacity: usize,
888 pins: HashMap<u64, PinState>,
889 next_pin_id: u64,
890}
891
892impl SegmentLog {
893 fn new(capacity: usize) -> Self {
894 SegmentLog {
895 entries: VecDeque::with_capacity(capacity),
896 base: 0,
897 published: 0,
898 capacity,
899 pins: HashMap::new(),
900 next_pin_id: 0,
901 }
902 }
903
904 /// Unconditional evict-then-push — exactly [`ClassLog::push`]'s shape.
905 /// [`ArchiveOverrun`] handling against `pins` happens *before* this is
906 /// called; see [`SegmentWriter::publish_segment`].
907 fn push(&mut self, entry: SegmentEntry) {
908 if self.entries.len() == self.capacity {
909 self.entries.pop_front();
910 self.base += 1;
911 }
912 self.entries.push_back(entry);
913 self.published += 1;
914 }
915}
916
917/// How one [`EventEntry`] is currently addressable on the trunk's 90 kHz
918/// absolute clock ([`timed_metadata::MediaTime`]) — the distinction
919/// architecture-audit finding B1 exists to make honest. See
920/// [The event log](self#the-event-log-90-khz-absolute-and-the-b1-crux) for
921/// why these three states cannot be collapsed into one `MediaTime` without
922/// reintroducing B1's silent-wrong-instant failure.
923#[derive(Debug, Clone, Copy, PartialEq, Eq)]
924#[non_exhaustive]
925pub enum EventAnchor {
926 /// Already expressible on this trunk's 90 kHz absolute clock — a
927 /// SCTE-35 `splice_time` PTS after [`timed_metadata::Timeline`]'s
928 /// 33-bit wrap-unroll, or an `emsg` v1 (already-absolute)
929 /// `presentation_time` on this same clock. The only variant
930 /// [`Trunk::events_between`]/[`Trunk::events_in_segment`] can ever
931 /// match against.
932 Media(MediaTime),
933 /// Segment-relative (`emsg` v0's `presentation_time_delta`, ISO/IEC
934 /// 23009-1 §5.10.3.3): this event's media time is `delta` ticks after
935 /// the *start* of segment `segment_number` — a start this entry does
936 /// not know yet. Resolves in place, to that segment's own reported
937 /// start, the instant [`SegmentWriter::note_segment_start`] reports it;
938 /// until then it stays exactly this variant — addressable by
939 /// `segment_number` (once a boundary exists), never by a fabricated
940 /// media time.
941 Segment {
942 /// The target segment's sequence number — matches
943 /// [`SegmentEntry::sequence_number`].
944 segment_number: u32,
945 /// `presentation_time_delta`: ticks after that segment's start.
946 delta: u64,
947 },
948 /// GPS/UTC wall-clock only (SCTE-35 `splice_schedule.utc_splice_time`,
949 /// §9.7.4): this event has **no** media-timeline position at all, only
950 /// an instant on the wall clock, until
951 /// [`SegmentWriter::set_time_anchor`] gives the event log a
952 /// [`TimeAnchor`] to translate through. **This is the B1 case** — see
953 /// [The event log](self#the-event-log-90-khz-absolute-and-the-b1-crux).
954 Utc {
955 /// Milliseconds since the Unix epoch — matches
956 /// [`TimeAnchor::utc_epoch_ms`]'s unit.
957 utc_epoch_ms: i64,
958 },
959}
960
961/// One entry in the event log: the owned, lossless [`TimedEvent`] this
962/// trunk carries verbatim — see
963/// [The event log](self#the-event-log-90-khz-absolute-and-the-b1-crux) for
964/// why this is *the* published `timed_metadata` type, not a parallel one —
965/// plus its current [`EventAnchor`] resolution state.
966#[derive(Debug, Clone)]
967#[non_exhaustive]
968pub struct EventEntry {
969 /// The canonical event, carried verbatim.
970 pub event: TimedEvent,
971 /// This entry's current resolution state.
972 pub anchor: EventAnchor,
973}
974
975/// The event log: a bounded, append-ordered log of [`EventEntry`] values,
976/// plus the two small resolution tables an [`EventAnchor::Segment`]/
977/// [`EventAnchor::Utc`] entry resolves against.
978///
979/// Evict-then-push shape identical to [`ClassLog`]/[`SegmentLog`] —
980/// `base`/`published` mean exactly the same thing here as there.
981struct EventLog {
982 entries: VecDeque<EventEntry>,
983 base: u64,
984 published: u64,
985 capacity: usize,
986 /// Recently-reported segment starts, in the order
987 /// [`SegmentWriter::note_segment_start`] received them (playlist order in
988 /// practice, since segments are announced in sequence). Bounded by the
989 /// **same** `capacity` as `entries` — see [`TrunkConfig::event_capacity`]'s
990 /// doc for why this deliberately is not a second, independently-tuned
991 /// knob.
992 segment_starts: VecDeque<(u32, MediaTime)>,
993 /// The one wall-clock↔media-clock mapping this trunk's event log
994 /// knows, if any. Mirrors [`timed_metadata::Timeline`]'s own
995 /// `anchor: Option<TimeAnchor>` field — one mapping per session/trunk,
996 /// not one per event.
997 time_anchor: Option<TimeAnchor>,
998}
999
1000impl EventLog {
1001 fn new(capacity: usize) -> Self {
1002 EventLog {
1003 entries: VecDeque::with_capacity(capacity),
1004 base: 0,
1005 published: 0,
1006 capacity,
1007 segment_starts: VecDeque::with_capacity(capacity),
1008 time_anchor: None,
1009 }
1010 }
1011
1012 /// Resolve `anchor` against whatever segment starts / time anchor are
1013 /// already known — **without** fabricating a resolution the log cannot
1014 /// yet justify. An anchor this call cannot resolve is returned
1015 /// unchanged: no anchor, no media time, per B1.
1016 fn try_resolve(&self, anchor: EventAnchor) -> EventAnchor {
1017 match anchor {
1018 EventAnchor::Segment {
1019 segment_number,
1020 delta,
1021 } => self
1022 .segment_starts
1023 .iter()
1024 .find(|(n, _)| *n == segment_number)
1025 .map(|(_, start)| EventAnchor::Media(MediaTime(start.0.saturating_add(delta))))
1026 .unwrap_or(anchor),
1027 EventAnchor::Utc { utc_epoch_ms } => self
1028 .time_anchor
1029 .as_ref()
1030 .map(|a| EventAnchor::Media(epoch_ms_to_media(a, utc_epoch_ms)))
1031 .unwrap_or(anchor),
1032 EventAnchor::Media(_) => anchor,
1033 }
1034 }
1035
1036 /// Push one event, evicting the oldest if the log is already at
1037 /// `capacity`. Never rejects, never blocks — exactly [`ClassLog::push`]/
1038 /// [`SegmentLog::push`]'s contract.
1039 fn push(&mut self, event: TimedEvent, anchor: EventAnchor) {
1040 let anchor = self.try_resolve(anchor);
1041 if self.entries.len() == self.capacity {
1042 self.entries.pop_front();
1043 self.base += 1;
1044 }
1045 self.entries.push_back(EventEntry { event, anchor });
1046 self.published += 1;
1047 }
1048
1049 /// Record segment `segment_number`'s start on this trunk's 90 kHz
1050 /// absolute clock, and resolve, **in place**, every still-pending
1051 /// [`EventAnchor::Segment`] entry that targets exactly this
1052 /// `segment_number` — not whichever segment happened to be open when
1053 /// the event was published (that would resolve to *a* segment, not
1054 /// *the* segment the `emsg` actually named, which is exactly the bug
1055 /// this design avoids).
1056 fn note_segment_start(&mut self, segment_number: u32, start: MediaTime) {
1057 if self.segment_starts.len() == self.capacity {
1058 self.segment_starts.pop_front();
1059 }
1060 self.segment_starts.push_back((segment_number, start));
1061 for entry in &mut self.entries {
1062 if let EventAnchor::Segment {
1063 segment_number: n,
1064 delta,
1065 } = entry.anchor
1066 && n == segment_number
1067 {
1068 entry.anchor = EventAnchor::Media(MediaTime(start.0.saturating_add(delta)));
1069 }
1070 }
1071 }
1072
1073 /// Record this trunk's wall-clock↔media-clock mapping, and resolve, in
1074 /// place, every still-pending [`EventAnchor::Utc`] entry through it.
1075 /// Before this call, a `Utc`-anchored entry stays a `Utc` entry — see
1076 /// [The event log](self#the-event-log-90-khz-absolute-and-the-b1-crux).
1077 fn set_time_anchor(&mut self, anchor: TimeAnchor) {
1078 self.time_anchor = Some(anchor);
1079 for entry in &mut self.entries {
1080 if let EventAnchor::Utc { utc_epoch_ms } = entry.anchor {
1081 entry.anchor = EventAnchor::Media(epoch_ms_to_media(&anchor, utc_epoch_ms));
1082 }
1083 }
1084 }
1085}
1086
1087/// The inverse of [`TimeAnchor::media_to_epoch_ms`]: the [`MediaTime`]
1088/// `anchor` implies for a UTC instant (milliseconds since the Unix epoch).
1089///
1090/// Plain affine algebra — the mirror image of a function `timed_metadata`
1091/// already publishes — **not** a reimplementation of
1092/// [`timed_metadata::Timeline`]'s 33-bit wrap-unroll, a different, modular
1093/// arithmetic problem this module does not re-solve; see
1094/// [The event log](self#the-event-log-90-khz-absolute-and-the-b1-crux).
1095/// Clamps rather than panics on an out-of-range result — a malformed or
1096/// adversarial `splice_schedule` entry must not crash the writer.
1097fn epoch_ms_to_media(anchor: &TimeAnchor, utc_epoch_ms: i64) -> MediaTime {
1098 let delta_ms = i128::from(utc_epoch_ms) - i128::from(anchor.utc_epoch_ms);
1099 let delta_ticks = delta_ms * i128::from(PTS_HZ) / 1000;
1100 let media = i128::from(anchor.pts_90k) + delta_ticks;
1101 MediaTime(media.clamp(0, i128::from(u64::MAX)) as u64)
1102}
1103
1104/// One LL-HLS **partial segment** ("part") of the segment currently being
1105/// written — RFC 8216bis §4.4.4.9's independently-fetchable CMAF chunk,
1106/// addressable by `(segment_number, part_index)` the way a client actually
1107/// asks for one (`_HLS_msn`/`_HLS_part`, or a `part-<seq>.<idx>.m4s` URI). See
1108/// [The live-part log](self#the-live-part-log-parts-before-their-segment-closes).
1109///
1110/// Does **not** reuse `transmux::ll_hls::PartInfo` whole, for the same reason
1111/// [`SegmentEntry`] does not reuse `transmux::ll_hls::SegmentInfo` whole:
1112/// `PartInfo::bytes` is `Vec<u8>`, and copying it into a `Bytes` here to get
1113/// zero-copy fan-out ([Zero-copy fan-out](self#zero-copy-fan-out-honestly))
1114/// would be exactly one copy per part, on the one path this module exists to
1115/// keep copy-free; a caller publishing a part therefore builds a
1116/// `bytes::Bytes` directly (e.g. from the encoder's own output buffer)
1117/// instead of routing through `Vec<u8>` first.
1118#[derive(Debug, Clone)]
1119#[non_exhaustive]
1120pub struct PartEntry {
1121 /// The part's encoded bytes: a bare `moof`+`mdat` CMAF fragment (no
1122 /// `styp`). `Bytes`, not `Vec<u8>` — fan-out to every reader of this
1123 /// entry is a refcount bump, not a copy; see
1124 /// [Zero-copy fan-out](self#zero-copy-fan-out-honestly).
1125 pub bytes: Bytes,
1126 /// The parent segment's sequence number — matches
1127 /// [`SegmentEntry::sequence_number`] once that segment closes.
1128 pub segment_number: u32,
1129 /// 0-based index of this part within its parent segment.
1130 pub part_index: u32,
1131 /// This part's duration, wall-clock.
1132 pub duration: Duration,
1133 /// `true` when this part's first sample is a sync sample, so it begins
1134 /// with an independently decodable frame (RFC 8216bis's
1135 /// `INDEPENDENT=YES`).
1136 pub independent: bool,
1137}
1138
1139impl PartEntry {
1140 /// Build one part log entry.
1141 pub fn new(
1142 bytes: impl Into<Bytes>,
1143 segment_number: u32,
1144 part_index: u32,
1145 duration: Duration,
1146 independent: bool,
1147 ) -> Self {
1148 PartEntry {
1149 bytes: bytes.into(),
1150 segment_number,
1151 part_index,
1152 duration,
1153 independent,
1154 }
1155 }
1156}
1157
1158/// The live-part log: a bounded, append-ordered log of [`PartEntry`] values.
1159///
1160/// Evict-then-push shape identical to [`ClassLog`]/[`SegmentLog`]/[`EventLog`]
1161/// — `base`/`published` mean exactly the same thing here as there. Unlike the
1162/// segment log, publishing a segment ([`SegmentWriter::publish_segment`]) does
1163/// **not** touch this ring at all — see
1164/// [The live-part log](self#the-live-part-log-parts-before-their-segment-closes)
1165/// for why a part's addressability deliberately does not change the instant
1166/// its parent segment closes.
1167struct PartLog {
1168 entries: VecDeque<PartEntry>,
1169 base: u64,
1170 published: u64,
1171 capacity: usize,
1172}
1173
1174impl PartLog {
1175 fn new(capacity: usize) -> Self {
1176 PartLog {
1177 entries: VecDeque::with_capacity(capacity),
1178 base: 0,
1179 published: 0,
1180 capacity,
1181 }
1182 }
1183
1184 /// Push one part, evicting the oldest if the log is already at
1185 /// `capacity`. Never rejects, never blocks — exactly [`ClassLog::push`]/
1186 /// [`SegmentLog::push`]/[`EventLog::push`]'s contract.
1187 fn push(&mut self, entry: PartEntry) {
1188 if self.entries.len() == self.capacity {
1189 self.entries.pop_front();
1190 self.base += 1;
1191 }
1192 self.entries.push_back(entry);
1193 self.published += 1;
1194 }
1195}
1196
1197/// The shared state behind one [`Trunk`]: the two sample [`ClassLog`]s, the
1198/// [`SegmentLog`], the [`EventLog`], and the [`PartLog`]. See
1199/// [The event log](self#the-event-log-90-khz-absolute-and-the-b1-crux) for
1200/// why the event log needed its own shape rather than being a third copy of
1201/// the other two, and
1202/// [The live-part log](self#the-live-part-log-parts-before-their-segment-closes)
1203/// for the fourth.
1204struct TrunkState {
1205 timed: ClassLog,
1206 sparse: ClassLog,
1207 segments: SegmentLog,
1208 events: EventLog,
1209 parts: PartLog,
1210 /// The program's current complete track set — see
1211 /// [`TrunkWriter::set_tracks`] for why this is always a full replacement
1212 /// snapshot, never a delta. `Arc<[TrackSpec]>` rather than a bare `Vec`
1213 /// so [`Trunk::tracks`] hands back a clone of the *reference*, not the
1214 /// whole set, to every caller — cheap even for a many-track program.
1215 tracks: Arc<[TrackSpec]>,
1216 /// Bumped by exactly one on every [`TrunkWriter::set_tracks`] call — see
1217 /// [`Trunk::track_generation`] for why a consumer compares this instead
1218 /// of the track [`Vec`] itself.
1219 track_generation: u64,
1220}
1221
1222/// The sample ring: bounded, dual-retention-class, single-writer,
1223/// multi-cursor. See the [module docs](self) for the design this
1224/// implements and the benchmark that shaped it.
1225///
1226/// Always held as `Arc<Trunk>` — [`Trunk::writer`] and [`Trunk::subscribe`]
1227/// take `self: &Arc<Self>` because a [`TrunkWriter`]/[`SampleCursor`] each
1228/// need to keep the shared state alive independently of the `Trunk` handle
1229/// that created them, exactly as `spikes/trunk-bench`'s validated shape
1230/// does.
1231pub struct Trunk {
1232 state: Mutex<TrunkState>,
1233 /// Wakes a [`SegmentWriter::publish_segment`] parked on
1234 /// [`ArchiveOverrun::StallIngest`] once a pin it is waiting on advances
1235 /// (a [`SegmentCursor::poll`] consuming further) or is released (its
1236 /// cursor dropped). Paired with `state` in the usual `Condvar` idiom:
1237 /// `wait` atomically releases the `Mutex` while parked, so a stalled
1238 /// segment publish does not hold the lock other `Trunk` operations
1239 /// (sample publish, any cursor's `poll`) need — see
1240 /// [The DVR contradiction](self#the-dvr-contradiction-losslessness-from-retention-not-back-pressure).
1241 segment_pin_released: Condvar,
1242 /// Guards [`Trunk::writer`]'s single-take — the **samples + events** ring
1243 /// group. See [One writer per ring group](self#one-writer-per-ring-group-not-one-writer-per-trunk).
1244 writer_taken: AtomicBool,
1245 /// Guards [`Trunk::segment_writer`]'s single-take — the
1246 /// **segments + parts** ring group, taken independently of
1247 /// `writer_taken` so a segmenter and the ingest driver can each hold
1248 /// their own write handle at once. See
1249 /// [One writer per ring group](self#one-writer-per-ring-group-not-one-writer-per-trunk).
1250 segment_writer_taken: AtomicBool,
1251 /// Broad "a part or a segment close was just published, go re-check
1252 /// your condition" notification — see
1253 /// [The reader-wake primitive](self#the-reader-wake-primitive-listen-not-one-registration-per-remote-peer).
1254 /// Bumped by exactly [`SegmentWriter::publish_part`]/
1255 /// [`SegmentWriter::publish_segment`] and, since the ingress track-set
1256 /// plumbing (issue #781), [`TrunkWriter::set_tracks`] — never by a
1257 /// sample/event publish (nothing today waits on those through this
1258 /// channel). Track-set changes are rare compared to samples/parts, so
1259 /// folding them into this same broad wake is additive scope, not a new
1260 /// channel to reason about.
1261 progress: Event,
1262 /// Count of currently-registered, not-yet-dropped [`ProgressListener`]s —
1263 /// what bounds [`Trunk::listen`] against `part_waiter_cap`. A plain
1264 /// `AtomicUsize`, not part of `state`'s `Mutex`, so registering/releasing
1265 /// a listener never contends the same lock `publish`/`poll` do.
1266 waiter_count: AtomicUsize,
1267 /// Copy of [`TrunkConfig::part_capacity`], read without locking `state` —
1268 /// the cap [`Trunk::listen`] enforces against `waiter_count`. See
1269 /// [The reader-wake primitive](self#the-reader-wake-primitive-listen-not-one-registration-per-remote-peer)
1270 /// for why this reuses `part_capacity` rather than adding a sixth,
1271 /// independent knob.
1272 part_waiter_cap: usize,
1273}
1274
1275impl Trunk {
1276 /// Construct a fresh, empty `Trunk`.
1277 ///
1278 /// Cannot fail and cannot panic on its configuration: every
1279 /// [`TrunkConfig`] capacity is a [`NonZeroUsize`], so the one invalid
1280 /// value (zero — a ring that evicts every entry the instant it is
1281 /// pushed) is unrepresentable rather than merely rejected. See
1282 /// [`TrunkConfig`]'s own docs for why that replaced this method's
1283 /// former five `assert!`s.
1284 pub fn new(config: TrunkConfig) -> Arc<Trunk> {
1285 Arc::new(Trunk {
1286 state: Mutex::new(TrunkState {
1287 timed: ClassLog::new(config.timed_capacity.get()),
1288 sparse: ClassLog::new(config.sparse_capacity.get()),
1289 segments: SegmentLog::new(config.segment_capacity.get()),
1290 events: EventLog::new(config.event_capacity.get()),
1291 parts: PartLog::new(config.part_capacity.get()),
1292 tracks: Arc::from(Vec::new()),
1293 track_generation: 0,
1294 }),
1295 segment_pin_released: Condvar::new(),
1296 writer_taken: AtomicBool::new(false),
1297 segment_writer_taken: AtomicBool::new(false),
1298 progress: Event::new(),
1299 waiter_count: AtomicUsize::new(0),
1300 part_waiter_cap: config.part_capacity.get(),
1301 })
1302 }
1303
1304 /// Take the one [`TrunkWriter`] for this `Trunk` — the write handle for
1305 /// the **samples + events** ring group ([`TrunkWriter::publish`]/
1306 /// [`TrunkWriter::publish_event`]). See
1307 /// [One writer per ring group](self#one-writer-per-ring-group-not-one-writer-per-trunk)
1308 /// for the invariant this enforces (and why it does not also cover
1309 /// [`Trunk::segment_writer`]'s group).
1310 ///
1311 /// Returns `None` on every call after the first — this ring group has
1312 /// exactly one writer, enforced here rather than left as a
1313 /// documented-only convention, because a second concurrent sample/event
1314 /// writer would silently interleave two unrelated publish sequences into
1315 /// the same ring with no way for a reader to tell them apart.
1316 pub fn writer(self: &Arc<Self>) -> Option<TrunkWriter> {
1317 self.writer_taken
1318 .compare_exchange(false, true, Ordering::AcqRel, Ordering::Acquire)
1319 .ok()
1320 .map(|_| TrunkWriter {
1321 trunk: Arc::clone(self),
1322 })
1323 }
1324
1325 /// Take the one [`SegmentWriter`] for this `Trunk` — the write handle for
1326 /// the **segments + parts** ring group ([`SegmentWriter::publish_segment`]/
1327 /// [`SegmentWriter::publish_part`]/[`SegmentWriter::note_segment_start`]/
1328 /// [`SegmentWriter::set_time_anchor`]), independent of [`Trunk::writer`]'s
1329 /// group so a segmenter can hold this while the ingest driver
1330 /// simultaneously holds a [`TrunkWriter`] — see
1331 /// [One writer per ring group](self#one-writer-per-ring-group-not-one-writer-per-trunk)
1332 /// for why the split is safe and what it does and does not guarantee
1333 /// across rings.
1334 ///
1335 /// Returns `None` on every call after the first — this ring group has
1336 /// exactly one writer too, guarded by its own `AtomicBool` rather than
1337 /// [`Trunk::writer`]'s, for exactly the same reason: a second concurrent
1338 /// segment/part writer would silently interleave two unrelated publish
1339 /// sequences into the segment or part ring.
1340 pub fn segment_writer(self: &Arc<Self>) -> Option<SegmentWriter> {
1341 self.segment_writer_taken
1342 .compare_exchange(false, true, Ordering::AcqRel, Ordering::Acquire)
1343 .ok()
1344 .map(|_| SegmentWriter {
1345 trunk: Arc::clone(self),
1346 })
1347 }
1348
1349 /// Subscribe a new [`SampleCursor`], starting from *now* — the next
1350 /// entry [`TrunkWriter::publish`] produces after this call, not any
1351 /// backlog already in either ring. See [`Trunk::subscribe_from_backlog`]
1352 /// for the seek-to-past variant this method's own docs used to
1353 /// anticipate (a consumer built *after* samples it needs already landed
1354 /// in the ring — e.g. a segmenter reacting to the same batch that
1355 /// announced its program — wants that one instead).
1356 ///
1357 /// # This call *is* fan-out — read this before calling it per connection
1358 ///
1359 /// `spikes/trunk-bench` measured writer cost as **O(N) in cursor
1360 /// count** (956 ns → 9.98 µs from 1 → 16 readers; spec §3.1) — every
1361 /// cursor contends the same shared lock every publish. **A cursor is
1362 /// for a distinct consumer of the stream** (a segmenter, a DVR writer,
1363 /// an analysis tap, one push relay) — **never** one per peer of a
1364 /// one-to-many protocol. Supported reader count is **single-digit by
1365 /// design**: LL-HLS serving a thousand viewers takes **one** cursor
1366 /// here and fans out to its viewers itself, at the layer that already
1367 /// holds per-viewer state anyway. Do not call this once per connection;
1368 /// there is no tee, broadcast channel, or per-consumer queue to reach
1369 /// for instead — a sample's payload is already [`bytes::Bytes`], so
1370 /// fan-out beyond this one cursor is a refcount bump the relay performs
1371 /// itself, not something this type needs to do for you.
1372 pub fn subscribe(self: &Arc<Self>) -> SampleCursor {
1373 let state = self.state.lock().expect("Trunk state lock poisoned");
1374 SampleCursor {
1375 trunk: Arc::clone(self),
1376 timed_consumed: state.timed.published,
1377 sparse_consumed: state.sparse.published,
1378 }
1379 }
1380
1381 /// Subscribe a new [`SampleCursor`], starting from the **oldest entry
1382 /// each ring currently retains** instead of [`Trunk::subscribe`]'s
1383 /// "now" — i.e. this cursor's first `poll` replays whatever backlog is
1384 /// still resident in the `Timed` ring and the `Sparse` ring, each
1385 /// independently, before catching up to the live tail.
1386 ///
1387 /// This is the "seek-to-past variant" [`Trunk::subscribe`]'s own docs
1388 /// anticipated ("a later step may add a seek-to-past variant... this
1389 /// step does not need one") — the step turned out to be issue #808's
1390 /// segment-bridge fix: a [`TrunkWriter::publish`] batch that lands
1391 /// *before* a consumer subscribes (e.g. the very same `feed` call that
1392 /// both announces a program and publishes its first samples) is
1393 /// otherwise invisible to a [`Trunk::subscribe`] cursor forever, even
1394 /// though the samples are sitting right there in the ring.
1395 ///
1396 /// # Replay is bounded by ring capacity, not "everything ever published"
1397 ///
1398 /// This does **not** reach further back than what each ring still
1399 /// holds: an entry already evicted by [`TrunkConfig::timed_capacity`]/
1400 /// [`TrunkConfig::sparse_capacity`] before this call is gone, exactly as
1401 /// it would be for any other cursor — there is no unbounded replay log
1402 /// behind this method, only the same fixed-size rings every other
1403 /// cursor reads. Concretely: this cursor starts at each ring's current
1404 /// `base` (the oldest index still resident), not index 0, so its first
1405 /// `poll` never reports a spurious `Lagged`/`Degraded` for data that was
1406 /// evicted *before* this call — from this cursor's point of view,
1407 /// "backlog" means "what the ring can show me right now", not "what was
1408 /// ever published". Both retention classes replay this way,
1409 /// independently: a `Timed` backlog and a `Sparse` backlog are each
1410 /// bounded by their own ring's own capacity.
1411 ///
1412 /// [`SampleCursorItem::Lagged`]/[`SampleCursorItem::Degraded`] still
1413 /// fire exactly as they do for a [`Trunk::subscribe`] cursor for any
1414 /// loss that happens **after** this call — falling behind the live tail
1415 /// once subscribed is reported in-band the same way for both kinds of
1416 /// cursor; only the starting position differs.
1417 ///
1418 /// # This call *is* fan-out — read this before calling it per connection
1419 ///
1420 /// Exactly [`Trunk::subscribe`]'s own fan-out warning, verbatim: writer
1421 /// cost is **O(N) in cursor count** (`spikes/trunk-bench` measured 956 ns
1422 /// → 9.98 µs from 1 → 16 readers; spec §3.1) — every cursor contends the
1423 /// same shared lock every publish. **A cursor is for a distinct
1424 /// consumer of the stream**, **never** one per peer of a one-to-many
1425 /// protocol. Supported reader count is **single-digit by design**; do
1426 /// not call this once per connection.
1427 pub fn subscribe_from_backlog(self: &Arc<Self>) -> SampleCursor {
1428 let state = self.state.lock().expect("Trunk state lock poisoned");
1429 SampleCursor {
1430 trunk: Arc::clone(self),
1431 timed_consumed: state.timed.base,
1432 sparse_consumed: state.sparse.base,
1433 }
1434 }
1435
1436 /// Diagnostic: entries currently resident in the `Timed` ring. Never
1437 /// exceeds [`TrunkConfig::timed_capacity`].
1438 pub fn timed_len(&self) -> usize {
1439 self.state
1440 .lock()
1441 .expect("Trunk state lock poisoned")
1442 .timed
1443 .entries
1444 .len()
1445 }
1446
1447 /// Diagnostic: entries currently resident in the `Sparse` ring. Never
1448 /// exceeds [`TrunkConfig::sparse_capacity`].
1449 pub fn sparse_len(&self) -> usize {
1450 self.state
1451 .lock()
1452 .expect("Trunk state lock poisoned")
1453 .sparse
1454 .entries
1455 .len()
1456 }
1457
1458 /// Subscribe a new **non-pinning** [`SegmentCursor`], starting from
1459 /// *now* — the same "next entry only, no backlog" rule as
1460 /// [`Trunk::subscribe`], and the same single-digit-reader,
1461 /// one-cursor-per-distinct-consumer guidance from that method's docs
1462 /// applies here verbatim (this cursor contends exactly the lock
1463 /// `subscribe`'s cursors do).
1464 ///
1465 /// This cursor is **not** protected by [`ArchiveOverrun`]: if it falls
1466 /// behind the segment log's ordinary [`TrunkConfig::segment_capacity`]
1467 /// eviction, it simply sees [`SegmentCursorItem::Lagged`], exactly like
1468 /// an ordinary [`RetentionClass::Timed`] sample reader. Use this for a
1469 /// consumer that tolerates ordinary loss (LL-HLS window rendering,
1470 /// catch-up within the live window) — use [`Trunk::pin_segments`]
1471 /// instead for a consumer that must not miss a segment (DVR/archive).
1472 pub fn subscribe_segments(self: &Arc<Self>) -> SegmentCursor {
1473 let state = self.state.lock().expect("Trunk state lock poisoned");
1474 SegmentCursor {
1475 trunk: Arc::clone(self),
1476 consumed: state.segments.published,
1477 pin_id: None,
1478 done: false,
1479 }
1480 }
1481
1482 /// Subscribe a new **pinning** [`SegmentCursor`] for a DVR/archive
1483 /// consumer that must not miss a segment — see
1484 /// [The DVR contradiction](self#the-dvr-contradiction-losslessness-from-retention-not-back-pressure)
1485 /// for the full design story this method is the entry point for.
1486 ///
1487 /// `on_overrun` is this cursor's chosen [`ArchiveOverrun`] for the one
1488 /// moment its guarantee runs out: the segment log at
1489 /// [`TrunkConfig::segment_capacity`], about to evict an entry this
1490 /// cursor has not yet consumed. There is no default parameter here on
1491 /// purpose — pinning is an explicit request for a stronger guarantee
1492 /// than [`Trunk::subscribe_segments`] gives, so the trade made when that
1493 /// guarantee cannot be kept is an explicit choice too, not a silent
1494 /// fallback (though [`ArchiveOverrun::default`] exists for a caller that
1495 /// affirmatively wants the same default the rest of this module uses).
1496 ///
1497 /// Also starts from *now*, and also single-digit-by-design — the same
1498 /// fan-out rule as [`Trunk::subscribe`] and [`Trunk::subscribe_segments`]
1499 /// applies; a pinning cursor is exactly as expensive per publish as any
1500 /// other.
1501 pub fn pin_segments(self: &Arc<Self>, on_overrun: ArchiveOverrun) -> SegmentCursor {
1502 let mut state = self.state.lock().expect("Trunk state lock poisoned");
1503 let pin_id = state.segments.next_pin_id;
1504 state.segments.next_pin_id += 1;
1505 let consumed = state.segments.published;
1506 state.segments.pins.insert(
1507 pin_id,
1508 PinState {
1509 consumed,
1510 policy: on_overrun,
1511 terminated: false,
1512 },
1513 );
1514 SegmentCursor {
1515 trunk: Arc::clone(self),
1516 consumed: 0,
1517 pin_id: Some(pin_id),
1518 done: false,
1519 }
1520 }
1521
1522 /// Diagnostic: entries currently resident in the segment log. Never
1523 /// exceeds [`TrunkConfig::segment_capacity`] — true even with an
1524 /// un-acking pinning cursor attached, which is exactly the property
1525 /// [The DVR contradiction](self#the-dvr-contradiction-losslessness-from-retention-not-back-pressure)'s
1526 /// "pinning is bounded" claim means.
1527 pub fn segment_len(&self) -> usize {
1528 self.state
1529 .lock()
1530 .expect("Trunk state lock poisoned")
1531 .segments
1532 .entries
1533 .len()
1534 }
1535
1536 /// Subscribe a new [`EventCursor`] over the event log, starting from
1537 /// *now* — the same "next entry only, no backlog" rule as
1538 /// [`Trunk::subscribe`]/[`Trunk::subscribe_segments`], and the same
1539 /// single-digit-reader, one-cursor-per-distinct-consumer guidance
1540 /// applies here verbatim (this cursor contends exactly the lock every
1541 /// other cursor does).
1542 ///
1543 /// A streaming consumer — e.g. a playback scheduler that wants every
1544 /// event as it resolves — wants this. A point-in-time query — "what has
1545 /// resolved for segment N" (a manifest renderer), or "what resolved
1546 /// between T1 and T2" (that same scheduler, replaying its window) —
1547 /// wants [`Trunk::events_in_segment`]/[`Trunk::events_between`] instead;
1548 /// both read the same log, just as a snapshot rather than a moving
1549 /// position. See
1550 /// [The event log](self#the-event-log-90-khz-absolute-and-the-b1-crux).
1551 pub fn subscribe_events(self: &Arc<Self>) -> EventCursor {
1552 let state = self.state.lock().expect("Trunk state lock poisoned");
1553 EventCursor {
1554 trunk: Arc::clone(self),
1555 consumed: state.events.published,
1556 }
1557 }
1558
1559 /// Every currently-**resolved** ([`EventAnchor::Media`]) event whose
1560 /// media time falls in the half-open range `[from, to)` — start
1561 /// inclusive, end exclusive. An entry still `Segment`/`Utc`-anchored
1562 /// never appears here: it has no honest media time yet, and
1563 /// fabricating one to satisfy this query would be exactly B1 — see
1564 /// [The event log](self#the-event-log-90-khz-absolute-and-the-b1-crux).
1565 pub fn events_between(&self, from: MediaTime, to: MediaTime) -> Vec<EventEntry> {
1566 let state = self.state.lock().expect("Trunk state lock poisoned");
1567 state
1568 .events
1569 .entries
1570 .iter()
1571 .filter(|e| matches!(e.anchor, EventAnchor::Media(t) if t.0 >= from.0 && t.0 < to.0))
1572 .cloned()
1573 .collect()
1574 }
1575
1576 /// Every currently-resolved event whose media time falls within segment
1577 /// `segment_number`'s span: `[start_N, start_{N+1})` once
1578 /// [`SegmentWriter::note_segment_start`] has reported the *next*
1579 /// segment's start too, else `[start_N, ∞)` (the segment is still open
1580 /// — nothing yet says where it ends). Returns nothing for a
1581 /// `segment_number` this trunk has never reported a start for: there is
1582 /// no span to contain anything, and an unresolved
1583 /// [`EventAnchor::Segment`] entry targeting it is not returned either,
1584 /// for the same B1 reason [`Trunk::events_between`] documents.
1585 pub fn events_in_segment(&self, segment_number: u32) -> Vec<EventEntry> {
1586 let state = self.state.lock().expect("Trunk state lock poisoned");
1587 let log = &state.events;
1588 let Some(&(_, start)) = log
1589 .segment_starts
1590 .iter()
1591 .find(|(n, _)| *n == segment_number)
1592 else {
1593 return Vec::new();
1594 };
1595 let end = log
1596 .segment_starts
1597 .iter()
1598 .find(|(n, _)| *n == segment_number + 1)
1599 .map(|&(_, s)| s.0);
1600 log.entries
1601 .iter()
1602 .filter(|e| match e.anchor {
1603 EventAnchor::Media(t) => t.0 >= start.0 && end.map(|e2| t.0 < e2).unwrap_or(true),
1604 _ => false,
1605 })
1606 .cloned()
1607 .collect()
1608 }
1609
1610 /// Diagnostic: entries currently resident in the event log. Never
1611 /// exceeds [`TrunkConfig::event_capacity`].
1612 pub fn event_len(&self) -> usize {
1613 self.state
1614 .lock()
1615 .expect("Trunk state lock poisoned")
1616 .events
1617 .entries
1618 .len()
1619 }
1620
1621 /// A live part's bytes by `(segment_number, part_index)` — the direct,
1622 /// `&self`-shaped query a [`ServedEgress`](crate::egress::ServedEgress)
1623 /// implementation needs to answer "does this part exist right now",
1624 /// exactly the shape [`Trunk::events_between`]/[`Trunk::events_in_segment`]
1625 /// already give the event log rather than forcing a caller to drain a
1626 /// cursor into a self-maintained cache. See
1627 /// [The live-part log](self#the-live-part-log-parts-before-their-segment-closes)
1628 /// for why a part answers `Some` here for as long as it has not been
1629 /// evicted by [`TrunkConfig::part_capacity`]'s ordinary bound —
1630 /// including after its parent segment has closed.
1631 pub fn part_bytes(&self, segment_number: u32, part_index: u32) -> Option<Bytes> {
1632 let state = self.state.lock().expect("Trunk state lock poisoned");
1633 state
1634 .parts
1635 .entries
1636 .iter()
1637 .find(|p| p.segment_number == segment_number && p.part_index == part_index)
1638 .map(|p| p.bytes.clone())
1639 }
1640
1641 /// Every currently-resident part of segment `segment_number`, in publish
1642 /// order — the part-log counterpart of [`Trunk::events_in_segment`],
1643 /// letting a caller derive "how many parts does the open segment have so
1644 /// far" (RFC 8216bis's `_HLS_part` blocking-reload condition) without a
1645 /// cursor.
1646 pub fn parts_in_segment(&self, segment_number: u32) -> Vec<PartEntry> {
1647 let state = self.state.lock().expect("Trunk state lock poisoned");
1648 state
1649 .parts
1650 .entries
1651 .iter()
1652 .filter(|p| p.segment_number == segment_number)
1653 .cloned()
1654 .collect()
1655 }
1656
1657 /// Diagnostic: entries currently resident in the live-part log. Never
1658 /// exceeds [`TrunkConfig::part_capacity`].
1659 pub fn part_len(&self) -> usize {
1660 self.state
1661 .lock()
1662 .expect("Trunk state lock poisoned")
1663 .parts
1664 .entries
1665 .len()
1666 }
1667
1668 /// Diagnostic: currently-outstanding [`ProgressListener`] registrations
1669 /// (from [`Trunk::listen`], not yet dropped). Never exceeds
1670 /// [`TrunkConfig::part_capacity`] — see
1671 /// [The reader-wake primitive](self#the-reader-wake-primitive-listen-not-one-registration-per-remote-peer).
1672 pub fn waiter_count(&self) -> usize {
1673 self.waiter_count.load(Ordering::Acquire)
1674 }
1675
1676 /// The sequence number of the most-recently-**closed** segment (the
1677 /// newest [`SegmentWriter::publish_segment`] call), or `None` if no
1678 /// segment has closed yet. Distinguishes "closed" (a whole, fetchable
1679 /// [`SegmentEntry`]) from merely "has live parts" — RFC 8216bis
1680 /// §6.2.5.2's bare-`_HLS_msn` blocking-reload condition needs exactly
1681 /// this distinction (mirrors
1682 /// `hls_runtime::server::MediaStore::last_closed_segment_seq`, which
1683 /// this method lets a `ServedEgress` stop duplicating).
1684 pub fn last_closed_segment(&self) -> Option<u32> {
1685 self.state
1686 .lock()
1687 .expect("Trunk state lock poisoned")
1688 .segments
1689 .entries
1690 .back()
1691 .map(|e| e.sequence_number)
1692 }
1693
1694 /// This program's current complete track set — see
1695 /// [`TrunkWriter::set_tracks`] for how it is set/replaced. Empty until
1696 /// the first `set_tracks` call (a freshly-minted `Trunk` announces no
1697 /// tracks yet). Stored as `Arc<[TrackSpec]>`, so this is a cheap `Arc`
1698 /// clone (a refcount bump), never a `Vec` copy, however many tracks the
1699 /// program carries.
1700 pub fn tracks(&self) -> Arc<[TrackSpec]> {
1701 Arc::clone(&self.state.lock().expect("Trunk state lock poisoned").tracks)
1702 }
1703
1704 /// Bumped by exactly one on every [`TrunkWriter::set_tracks`] call
1705 /// (including one that happens to set an identical set to what was
1706 /// already there — this counts *calls*, not distinct sets). Lets a
1707 /// consumer detect "the track set may have changed" by comparing two
1708 /// `u64`s rather than diffing two `Vec<TrackSpec>`s — cheap regardless
1709 /// of how many tracks a program carries. `0` until the first
1710 /// `set_tracks` call.
1711 pub fn track_generation(&self) -> u64 {
1712 self.state
1713 .lock()
1714 .expect("Trunk state lock poisoned")
1715 .track_generation
1716 }
1717
1718 /// Register for the next part/segment-close notification — see
1719 /// [The reader-wake primitive](self#the-reader-wake-primitive-listen-not-one-registration-per-remote-peer).
1720 ///
1721 /// Returns `None` once [`TrunkConfig::part_capacity`] concurrent
1722 /// registrations are already outstanding — the caller must not wait in
1723 /// that case (there is no slot to wait *in*); it should fall back to an
1724 /// immediate re-poll or answer its request as unavailable now, exactly
1725 /// as a caller must once [`crate::egress::AwaitPolicy`] itself has
1726 /// expired. **Register before re-checking the condition you are waiting
1727 /// on** — `event_listener`'s standard idiom, and the same ordering
1728 /// `hls_runtime::server::MediaStore::listen`'s own docs require —
1729 /// otherwise a `notify` racing your check can be missed.
1730 pub fn listen(self: &Arc<Self>) -> Option<ProgressListener> {
1731 loop {
1732 let current = self.waiter_count.load(Ordering::Acquire);
1733 if current >= self.part_waiter_cap {
1734 return None;
1735 }
1736 if self
1737 .waiter_count
1738 .compare_exchange(current, current + 1, Ordering::AcqRel, Ordering::Acquire)
1739 .is_ok()
1740 {
1741 break;
1742 }
1743 }
1744 Some(ProgressListener {
1745 _slot: WaiterSlot(Arc::clone(self)),
1746 listener: self.progress.listen(),
1747 })
1748 }
1749}
1750
1751/// RAII release of one [`Trunk`] waiter slot — split out from
1752/// [`ProgressListener`] itself (rather than a `Drop` impl directly on
1753/// `ProgressListener`) specifically so [`ProgressListener::wait_deadline`]
1754/// can destructure `self` and move its `listener` field into
1755/// [`event_listener::Listener::wait_deadline`] by value: Rust forbids moving
1756/// a field out of a type that implements `Drop` itself, but does not forbid
1757/// it for a type that merely *contains* a field whose type implements
1758/// `Drop` — each field is then dropped independently, in this case when the
1759/// destructured local bindings go out of scope at the end of that method.
1760struct WaiterSlot(Arc<Trunk>);
1761
1762impl Drop for WaiterSlot {
1763 /// Release this `Trunk`'s bounded waiter slot — see
1764 /// [The reader-wake primitive](self#the-reader-wake-primitive-listen-not-one-registration-per-remote-peer)
1765 /// for why this cap exists at all (an unbounded waiter set is a remote
1766 /// resource-exhaustion vector). Fires whether the owning
1767 /// [`ProgressListener`] was ever polled/waited on, woken, or simply
1768 /// dropped un-awaited — a caller that gives up on its own request must
1769 /// not leak a slot.
1770 fn drop(&mut self) {
1771 self.0.waiter_count.fetch_sub(1, Ordering::AcqRel);
1772 }
1773}
1774
1775/// A registered wake-up from [`Trunk::listen`] — the
1776/// [`event_listener::EventListener`] a [`ServedEgress`](crate::egress::ServedEgress)
1777/// adapter waits on, so a blocked request need not busy-poll. See
1778/// [The reader-wake primitive](self#the-reader-wake-primitive-listen-not-one-registration-per-remote-peer).
1779///
1780/// Releases this `Trunk`'s bounded waiter slot when dropped (via the
1781/// `_slot` field's own `Drop`, see this module's internal `WaiterSlot`) — whether this listener
1782/// was woken, timed out, or is simply discarded — so a caller that gives up
1783/// does not leak a slot forever.
1784pub struct ProgressListener {
1785 _slot: WaiterSlot,
1786 listener: EventListener,
1787}
1788
1789impl ProgressListener {
1790 /// Block the calling thread until woken or `deadline` passes, whichever
1791 /// comes first — `true` if woken, `false` on timeout. Composes with
1792 /// [`crate::egress::AwaitPolicy`]'s deadline: convert
1793 /// `AwaitPolicy::deadline` (a [`Timestamp`]) to the `std::time::Instant`
1794 /// your caller already anchors its `Timestamp`s to (see
1795 /// [`Timestamp::from_instant`]'s inverse — the caller holds the base
1796 /// `Instant` it built its `Timestamp`s from) and pass that here, so this
1797 /// call can never park past the caller's own bound.
1798 ///
1799 /// Deliberately no unbounded `wait()` is exposed here — only this
1800 /// deadline-bound form and the `Future` impl below (whose bound is
1801 /// whatever timeout the caller's own executor wraps it in, exactly the
1802 /// `hls_runtime::server` "caller-driven wait loop" shape) — matching
1803 /// this crate's [`crate::egress::AwaitPolicy`] philosophy that a wait on
1804 /// remote-triggerable input must never be able to park forever.
1805 pub fn wait_deadline(self, deadline: std::time::Instant) -> bool {
1806 // Destructuring here (not a method call on `self.listener` while
1807 // `self` stays intact) is exactly what requires `ProgressListener`
1808 // itself to carry no `Drop` impl — see `WaiterSlot`'s own doc.
1809 let ProgressListener { _slot, listener } = self;
1810 listener.wait_deadline(deadline).is_some()
1811 }
1812}
1813
1814impl Future for ProgressListener {
1815 type Output = ();
1816
1817 fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
1818 // `EventListener` is `Unpin` (event_listener's own guarantee), and so
1819 // is `WaiterSlot` (an `Arc` newtype), so `ProgressListener` as a
1820 // whole is `Unpin` too — safe to reach the inner listener through a
1821 // plain `&mut` and poll it directly.
1822 let this = self.get_mut();
1823 Pin::new(&mut this.listener).poll(cx)
1824 }
1825}
1826
1827/// The write handle for a [`Trunk`]'s **samples + events** ring group.
1828/// Obtained via [`Trunk::writer`]. See
1829/// [One writer per ring group](self#one-writer-per-ring-group-not-one-writer-per-trunk)
1830/// for why this group is exactly these two rings, and
1831/// [`SegmentWriter`] for the sibling handle covering segments + parts.
1832///
1833/// `publish` never blocks and never rejects: a full class ring evicts its
1834/// oldest entry (see the internal per-class log's push logic) rather than waiting for a reader or
1835/// erroring, so ingest never stalls because some [`SampleCursor`] is slow —
1836/// the same non-blocking-producer principle as [`crate::byte_tap::ByteTap::record`],
1837/// for the same reason (a broadcast head-end does not pause live ingest for
1838/// a lagging analysis tap or a stalled egress peer).
1839///
1840/// "Never blocks" describes the absence of any wait-for-a-reader code path,
1841/// not a claim that the underlying `Mutex` critical section is instant —
1842/// `publish` briefly contends the same lock [`SampleCursor::poll`] does, a
1843/// bounded amount of work independent of how far behind any reader is (this
1844/// is exactly what `spikes/trunk-bench` measured as the O(N)-in-cursor-count
1845/// cost, not an unbounded wait).
1846pub struct TrunkWriter {
1847 trunk: Arc<Trunk>,
1848}
1849
1850impl TrunkWriter {
1851 /// Publish one sample for `track_id` under `retention`.
1852 pub fn publish(&self, track_id: u32, retention: RetentionClass, sample: Sample) {
1853 let mut state = self.trunk.state.lock().expect("Trunk state lock poisoned");
1854 match retention {
1855 RetentionClass::Timed => state.timed.push(track_id, sample),
1856 RetentionClass::Sparse => state.sparse.push(track_id, sample),
1857 }
1858 }
1859
1860 /// Publish one event. Never blocks and never rejects — a full event log
1861 /// evicts its oldest entry exactly like the sample/segment logs.
1862 /// `anchor` is resolved immediately against whatever segment starts /
1863 /// time anchor this trunk already knows; if it cannot be resolved yet,
1864 /// the entry is stored exactly as given, and resolves later, in place,
1865 /// once [`SegmentWriter::note_segment_start`]/[`SegmentWriter::set_time_anchor`]
1866 /// supplies what was missing. See
1867 /// [The event log](self#the-event-log-90-khz-absolute-and-the-b1-crux).
1868 pub fn publish_event(&self, event: TimedEvent, anchor: EventAnchor) {
1869 let mut state = self.trunk.state.lock().expect("Trunk state lock poisoned");
1870 state.events.push(event, anchor);
1871 }
1872
1873 /// Replace this program's track set wholesale — the write side of
1874 /// [`Trunk::tracks`]/[`Trunk::track_generation`], and the method
1875 /// [`crate::ingress::IngestDriver`] calls to seed a freshly-minted
1876 /// `Trunk` from `SessionEvent::NewProgram`'s `tracks` and to apply a
1877 /// later `SessionEvent::TracksChanged`.
1878 ///
1879 /// `tracks` is taken as the **complete replacement set**, matching
1880 /// `SessionEvent::TracksChanged`'s own doc: a PMT (or any container's
1881 /// track-declaration mechanism) carries the whole elementary-stream
1882 /// list, so this call is idempotent (calling it twice with the same set
1883 /// leaves the trunk's tracks unchanged in content, only `track_generation`
1884 /// advances) and immune to delta-ordering bugs — there is no "add
1885 /// track"/"remove track" pair to apply out of order. A caller that wants
1886 /// to know *which* track changed diffs the previous [`Trunk::tracks`]
1887 /// snapshot against this one itself.
1888 ///
1889 /// Bumps [`Trunk::track_generation`] by exactly one and wakes any
1890 /// [`Trunk::listen`] registration, the same
1891 /// [`event_listener::Event::notify`] fan-out
1892 /// [`SegmentWriter::publish_segment`]/[`SegmentWriter::publish_part`]
1893 /// already use — see [`Trunk`]'s `progress` field doc for why a
1894 /// track-set change is folded into that same broad wake rather than a
1895 /// new channel.
1896 pub fn set_tracks(&self, tracks: Vec<TrackSpec>) {
1897 let mut state = self.trunk.state.lock().expect("Trunk state lock poisoned");
1898 state.tracks = Arc::from(tracks);
1899 state.track_generation += 1;
1900 drop(state);
1901 self.trunk.progress.notify(usize::MAX);
1902 }
1903}
1904
1905/// The write handle for a [`Trunk`]'s **segments + parts** ring group.
1906/// Obtained via [`Trunk::segment_writer`], independently of [`TrunkWriter`]
1907/// (via [`Trunk::writer`]) — see
1908/// [One writer per ring group](self#one-writer-per-ring-group-not-one-writer-per-trunk)
1909/// for why this split exists, why `note_segment_start`/`set_time_anchor` are
1910/// grouped here rather than on [`TrunkWriter`], and what is (and is not)
1911/// guaranteed about ordering relative to the sample/event rings.
1912///
1913/// Like [`TrunkWriter`], every method here either never blocks (ordinary
1914/// eviction, exactly the sample rings' non-blocking-producer principle) or
1915/// blocks only in the one documented [`ArchiveOverrun::StallIngest`] case —
1916/// see [`SegmentWriter::publish_segment`].
1917pub struct SegmentWriter {
1918 trunk: Arc<Trunk>,
1919}
1920
1921impl SegmentWriter {
1922 /// Publish one finished segment, in playlist order.
1923 ///
1924 /// Never blocks and never rejects for **every non-pinning**
1925 /// [`SegmentCursor`] and for every pinning cursor using
1926 /// [`ArchiveOverrun::Gap`] (the default) or [`ArchiveOverrun::Terminate`]
1927 /// — a full segment log evicts its oldest entry exactly like
1928 /// [`TrunkWriter::publish`]'s sample rings. The **one** exception, by
1929 /// design, is a pinning cursor using [`ArchiveOverrun::StallIngest`]
1930 /// that has not yet consumed the entry about to be evicted: this call
1931 /// blocks until that cursor consumes further (or is dropped) — see
1932 /// [The DVR contradiction](self#the-dvr-contradiction-losslessness-from-retention-not-back-pressure).
1933 /// The block is a [`std::sync::Condvar::wait`], which releases the
1934 /// shared `Mutex` while parked, so [`TrunkWriter::publish`] and every
1935 /// cursor's `poll` on *other* data remain free to proceed even while
1936 /// this call is stalled.
1937 ///
1938 /// Does **not** touch the live-part log — see
1939 /// [The live-part log](self#the-live-part-log-parts-before-their-segment-closes)
1940 /// for why a segment closing deliberately leaves that segment's parts
1941 /// exactly as addressable as they were the instant before. Wakes any
1942 /// [`Trunk::listen`] registration once this call is about to return
1943 /// (bare-`_HLS_msn` blocking-reload's condition), even on the
1944 /// `StallIngest` path — a waiter is woken only after the entry has
1945 /// actually landed, never merely because a pin released.
1946 pub fn publish_segment(&self, entry: SegmentEntry) {
1947 let mut state = self.trunk.state.lock().expect("Trunk state lock poisoned");
1948 loop {
1949 if state.segments.entries.len() < state.segments.capacity {
1950 // Room to push without evicting anything: no pin can be at
1951 // risk this round.
1952 break;
1953 }
1954 let oldest = state.segments.base;
1955 let mut must_wait = false;
1956 for pin in state.segments.pins.values_mut() {
1957 if pin.terminated || pin.consumed > oldest {
1958 // Either already given up on (Terminate already fired),
1959 // or this pin has already consumed the entry about to be
1960 // evicted — not at risk.
1961 continue;
1962 }
1963 match pin.policy {
1964 // Nothing to do here: eviction proceeds, and the owning
1965 // cursor's own `poll` reports the loss as `Gap` the same
1966 // way a non-pinning cursor's `poll` reports it as
1967 // ordinary `Lagged` — both read `base` vs. their own
1968 // progress, after the fact.
1969 ArchiveOverrun::Gap => {}
1970 ArchiveOverrun::Terminate => pin.terminated = true,
1971 ArchiveOverrun::StallIngest => must_wait = true,
1972 }
1973 }
1974 if !must_wait {
1975 break;
1976 }
1977 state = self
1978 .trunk
1979 .segment_pin_released
1980 .wait(state)
1981 .expect("Trunk segment_pin_released condvar poisoned");
1982 // Loop back around: re-check capacity/oldest/pins after waking —
1983 // the pin that was blocking may have advanced, been dropped, or
1984 // (if a *different* pin also needed this entry) still be
1985 // pending.
1986 }
1987 state.segments.push(entry);
1988 drop(state);
1989 self.trunk.progress.notify(usize::MAX);
1990 }
1991
1992 /// Publish one live part of the segment currently being written — see
1993 /// [The live-part log](self#the-live-part-log-parts-before-their-segment-closes).
1994 ///
1995 /// Never blocks and never rejects: a full part log evicts its oldest
1996 /// entry exactly like every other ring in this module — the same
1997 /// non-blocking-producer principle as [`TrunkWriter::publish`]/
1998 /// [`SegmentWriter::publish_segment`]'s ordinary (non-`StallIngest`) path.
1999 /// Wakes any [`Trunk::listen`] registration once this part has actually
2000 /// landed (RFC 8216bis blocking-reload's part-availability condition).
2001 pub fn publish_part(&self, entry: PartEntry) {
2002 let mut state = self.trunk.state.lock().expect("Trunk state lock poisoned");
2003 state.parts.push(entry);
2004 drop(state);
2005 self.trunk.progress.notify(usize::MAX);
2006 }
2007
2008 /// Report that segment `segment_number` starts at `start` on this
2009 /// trunk's 90 kHz absolute clock — the boundary an
2010 /// [`EventAnchor::Segment`] (an `emsg` v0's `presentation_time_delta`)
2011 /// needs before it can resolve. Called by whoever owns segmentation —
2012 /// the entity the spec's B1 fix names explicitly: "it cannot be
2013 /// finalised until the segmenter owns a boundary." Lives here, not on
2014 /// [`TrunkWriter`], for exactly that reason: only the segmenter can
2015 /// honestly report it. This does **not** append to the event ring — it
2016 /// resolves an already-published [`EventAnchor::Segment`] entry in
2017 /// place, so grouping it with [`SegmentWriter::publish_segment`] does not
2018 /// create a second appender for [`TrunkWriter::publish_event`]'s ring;
2019 /// see [One writer per ring group](self#one-writer-per-ring-group-not-one-writer-per-trunk).
2020 pub fn note_segment_start(&self, segment_number: u32, start: MediaTime) {
2021 let mut state = self.trunk.state.lock().expect("Trunk state lock poisoned");
2022 state.events.note_segment_start(segment_number, start);
2023 }
2024
2025 /// Give the event log a wall-clock↔media-clock mapping. Resolves every
2026 /// currently-pending [`EventAnchor::Utc`] entry immediately, and every
2027 /// future one at publish time, until a later call replaces it. Grouped
2028 /// with [`SegmentWriter::note_segment_start`] rather than split onto
2029 /// [`TrunkWriter`] — see
2030 /// [One writer per ring group](self#one-writer-per-ring-group-not-one-writer-per-trunk)
2031 /// for why, and the same in-place-resolution reasoning: this is not an
2032 /// append to the event ring either.
2033 pub fn set_time_anchor(&self, anchor: TimeAnchor) {
2034 let mut state = self.trunk.state.lock().expect("Trunk state lock poisoned");
2035 state.events.set_time_anchor(anchor);
2036 }
2037}
2038
2039/// One item [`SampleCursor::poll`] can hand back: data from either retention
2040/// class, or a loss report.
2041///
2042/// `#[non_exhaustive]`: this is the growth point for anything a cursor might
2043/// need to surface beyond "sample" or "loss" later, without a breaking
2044/// change to every match arm in the workspace.
2045#[derive(Debug, Clone)]
2046#[non_exhaustive]
2047pub enum SampleCursorItem {
2048 /// A [`RetentionClass::Timed`] sample for `track_id`.
2049 Timed {
2050 /// The publishing track.
2051 track_id: u32,
2052 /// The sample itself. Cloned from the ring's stored copy —
2053 /// `Sample.data: Bytes` is shared, not copied; see
2054 /// [Zero-copy fan-out](self#zero-copy-fan-out-honestly).
2055 sample: Sample,
2056 },
2057 /// A [`RetentionClass::Sparse`] sample for `track_id`.
2058 Sparse {
2059 /// The publishing track.
2060 track_id: u32,
2061 /// The sample itself; see the `Timed` variant's doc for the
2062 /// zero-copy note.
2063 sample: Sample,
2064 },
2065 /// This cursor fell behind the `Timed` ring: `skipped` entries were
2066 /// evicted before it read them. Ordinary loss — resume from the next
2067 /// sample; see [`RetentionClass::Timed`].
2068 Lagged {
2069 /// Exact count of `Timed` entries evicted since this cursor's last
2070 /// successful read of that class.
2071 skipped: u64,
2072 },
2073 /// This cursor fell behind the `Sparse` ring: `skipped` entries were
2074 /// evicted before it read them. **Not** ordinary loss — the consumer's
2075 /// derived state (e.g. splice-point tracking) is now wrong, not merely
2076 /// gapped; see [`RetentionClass::Sparse`] for what a consumer is
2077 /// expected to do about it.
2078 Degraded {
2079 /// Exact count of `Sparse` entries evicted since this cursor's last
2080 /// successful read of that class.
2081 skipped: u64,
2082 },
2083}
2084
2085/// A subscribed reader of a [`Trunk`]'s sample ring. Obtained via
2086/// [`Trunk::subscribe`] — **read that method's docs before creating more
2087/// than a handful of these.**
2088pub struct SampleCursor {
2089 trunk: Arc<Trunk>,
2090 /// How many `Timed` entries this cursor has consumed (returned via
2091 /// `poll`, or accounted for via a reported `Lagged`) since it
2092 /// subscribed. Compared against the shared `ClassLog::base` to detect
2093 /// loss — the same technique as `spikes/trunk-bench`'s `Cursor::read_seq`
2094 /// vs. `TrunkInner::base_seq`.
2095 timed_consumed: u64,
2096 /// The `Sparse`-class equivalent of `timed_consumed`.
2097 sparse_consumed: u64,
2098}
2099
2100impl SampleCursor {
2101 /// Pull the next item, if any is ready.
2102 ///
2103 /// Loss is always reported before further data, in the same
2104 /// `Option<SampleCursorItem>` as real samples — following
2105 /// [`crate::byte_tap::TapItem`]'s precedent: a consumer cannot poll past
2106 /// a `Lagged`/`Degraded` report to reach the data that follows a gap,
2107 /// because there is no side channel it could forget to check instead.
2108 ///
2109 /// # Merge order across the two retention classes
2110 ///
2111 /// A pending `Timed` lag report is checked first, then a pending
2112 /// `Sparse` lag report, then a ready `Sparse` sample, then a ready
2113 /// `Timed` sample. This gives **no cross-class chronological interleave
2114 /// guarantee** (see the [module docs](self) for why that is not
2115 /// something anything downstream needs) — only that, within each
2116 /// class, entries are returned in the exact order
2117 /// [`TrunkWriter::publish`] produced them, with no duplication and no
2118 /// unreported loss.
2119 pub fn poll(&mut self) -> Option<SampleCursorItem> {
2120 let state = self.trunk.state.lock().expect("Trunk state lock poisoned");
2121
2122 if self.timed_consumed < state.timed.base {
2123 let skipped = state.timed.base - self.timed_consumed;
2124 self.timed_consumed = state.timed.base;
2125 return Some(SampleCursorItem::Lagged { skipped });
2126 }
2127 if self.sparse_consumed < state.sparse.base {
2128 let skipped = state.sparse.base - self.sparse_consumed;
2129 self.sparse_consumed = state.sparse.base;
2130 return Some(SampleCursorItem::Degraded { skipped });
2131 }
2132
2133 let sparse_idx = (self.sparse_consumed - state.sparse.base) as usize;
2134 if let Some((track_id, sample)) = state.sparse.entries.get(sparse_idx) {
2135 self.sparse_consumed += 1;
2136 return Some(SampleCursorItem::Sparse {
2137 track_id: *track_id,
2138 sample: sample.clone(),
2139 });
2140 }
2141
2142 let timed_idx = (self.timed_consumed - state.timed.base) as usize;
2143 if let Some((track_id, sample)) = state.timed.entries.get(timed_idx) {
2144 self.timed_consumed += 1;
2145 return Some(SampleCursorItem::Timed {
2146 track_id: *track_id,
2147 sample: sample.clone(),
2148 });
2149 }
2150
2151 None
2152 }
2153}
2154
2155/// One item [`SegmentCursor::poll`] can hand back: a finished segment, or a
2156/// loss report.
2157///
2158/// `#[non_exhaustive]`: this is the growth point for anything a segment
2159/// cursor might need to surface beyond "segment" or "loss" later, without a
2160/// breaking change to every match arm in the workspace.
2161#[derive(Debug, Clone)]
2162#[non_exhaustive]
2163pub enum SegmentCursorItem {
2164 /// One finished segment, in playlist order.
2165 Segment(SegmentEntry),
2166 /// A **non-pinning** cursor (from [`Trunk::subscribe_segments`]) fell
2167 /// behind the segment log's ordinary [`TrunkConfig::segment_capacity`]
2168 /// eviction: `skipped` segments were evicted before it read them.
2169 /// Ordinary loss, exactly [`SampleCursorItem::Lagged`]'s contract —
2170 /// resume from the next segment.
2171 Lagged {
2172 /// Exact count of segments evicted since this cursor's last
2173 /// successful read.
2174 skipped: u64,
2175 },
2176 /// A **pinning** cursor's (from [`Trunk::pin_segments`])
2177 /// [`ArchiveOverrun::Gap`] policy fired: the log evicted `skipped`
2178 /// segments this cursor had not yet consumed rather than let its pin
2179 /// grow retention without bound. Unlike `Lagged`, this is the defect a
2180 /// DVR consumer must record as a hole in the archive — see
2181 /// [The DVR contradiction](self#the-dvr-contradiction-losslessness-from-retention-not-back-pressure).
2182 Gap {
2183 /// Exact count of segments evicted out from under this cursor's
2184 /// pin.
2185 skipped: u64,
2186 },
2187 /// This **pinning** cursor's [`ArchiveOverrun::Terminate`] policy fired:
2188 /// the log dropped its pin instead of gapping the recording or
2189 /// stalling ingest. This is the last item this cursor will ever yield —
2190 /// every `poll` after this one returns `None`.
2191 Terminated,
2192}
2193
2194/// A subscribed reader of a [`Trunk`]'s segment log. Obtained via
2195/// [`Trunk::subscribe_segments`] (ordinary, lossy-on-overflow) or
2196/// [`Trunk::pin_segments`] (pinning, [`ArchiveOverrun`]-governed) — **read
2197/// those methods' docs, and [The DVR contradiction](self#the-dvr-contradiction-losslessness-from-retention-not-back-pressure),
2198/// before creating more than a handful of these.**
2199pub struct SegmentCursor {
2200 trunk: Arc<Trunk>,
2201 /// Read progress for a **non-pinning** cursor (`pin_id.is_none()`) —
2202 /// exactly [`SampleCursor`]'s local `*_consumed` fields. Unused (and left
2203 /// at `0`) for a pinning cursor, whose progress instead lives in the
2204 /// shared `PinState::consumed` the writer must be able to see; see
2205 /// [`SegmentLog`].
2206 consumed: u64,
2207 /// `Some(id)` for a pinning cursor — the key into
2208 /// `TrunkState::segments.pins` this cursor's progress and policy are
2209 /// recorded under. `None` for an ordinary [`Trunk::subscribe_segments`]
2210 /// cursor.
2211 pin_id: Option<u64>,
2212 /// Set once this cursor has reported [`SegmentCursorItem::Terminated`] —
2213 /// every `poll` after that returns `None` rather than re-reporting it or
2214 /// resuming as if nothing happened.
2215 done: bool,
2216}
2217
2218impl SegmentCursor {
2219 /// Pull the next item, if any is ready.
2220 ///
2221 /// Loss is always reported before further data, in the same
2222 /// `Option<SegmentCursorItem>` as real segments — the same
2223 /// cannot-be-skipped-past precedent as [`SampleCursor::poll`]/
2224 /// [`crate::byte_tap::TapItem`].
2225 pub fn poll(&mut self) -> Option<SegmentCursorItem> {
2226 if self.done {
2227 return None;
2228 }
2229
2230 let Some(pin_id) = self.pin_id else {
2231 // Non-pinning: local `consumed`, exactly `SampleCursor::poll`'s
2232 // shape, against the one segment log instead of two class rings.
2233 let state = self.trunk.state.lock().expect("Trunk state lock poisoned");
2234 if self.consumed < state.segments.base {
2235 let skipped = state.segments.base - self.consumed;
2236 self.consumed = state.segments.base;
2237 return Some(SegmentCursorItem::Lagged { skipped });
2238 }
2239 let idx = (self.consumed - state.segments.base) as usize;
2240 return if let Some(entry) = state.segments.entries.get(idx) {
2241 self.consumed += 1;
2242 Some(SegmentCursorItem::Segment(entry.clone()))
2243 } else {
2244 None
2245 };
2246 };
2247
2248 // Pinning: progress lives in the shared `PinState`, because
2249 // `SegmentWriter::publish_segment` has to consult it before evicting,
2250 // not merely react to it afterward.
2251 let mut state = self.trunk.state.lock().expect("Trunk state lock poisoned");
2252 let Some(pin) = state.segments.pins.get(&pin_id) else {
2253 // Already removed (defensive: `Drop`/prior `Terminated` report
2254 // should make this unreachable in practice) — treat as done.
2255 self.done = true;
2256 return None;
2257 };
2258 if pin.terminated {
2259 state.segments.pins.remove(&pin_id);
2260 self.pin_id = None;
2261 self.done = true;
2262 return Some(SegmentCursorItem::Terminated);
2263 }
2264 let consumed = pin.consumed;
2265 if consumed < state.segments.base {
2266 let skipped = state.segments.base - consumed;
2267 state
2268 .segments
2269 .pins
2270 .get_mut(&pin_id)
2271 .expect("pin_id was resolved from this same locked state, so its entry exists")
2272 .consumed = state.segments.base;
2273 drop(state);
2274 // A pin advancing can free a `StallIngest` writer waiting on
2275 // exactly this pin.
2276 self.trunk.segment_pin_released.notify_all();
2277 return Some(SegmentCursorItem::Gap { skipped });
2278 }
2279 let idx = (consumed - state.segments.base) as usize;
2280 if let Some(entry) = state.segments.entries.get(idx) {
2281 let item = entry.clone();
2282 state
2283 .segments
2284 .pins
2285 .get_mut(&pin_id)
2286 .expect("pin_id was resolved from this same locked state, so its entry exists")
2287 .consumed += 1;
2288 drop(state);
2289 self.trunk.segment_pin_released.notify_all();
2290 return Some(SegmentCursorItem::Segment(item));
2291 }
2292 None
2293 }
2294}
2295
2296impl Drop for SegmentCursor {
2297 /// Release this cursor's pin, if it has one, so a dropped/abandoned
2298 /// pinning cursor cannot hold retention open (or a `StallIngest` writer
2299 /// blocked) forever — the same "a dead consumer must not grow memory
2300 /// without limit" guarantee as an actively-`Gap`-ping cursor, for the
2301 /// case where the consumer disappeared instead of choosing a policy.
2302 fn drop(&mut self) {
2303 if let Some(pin_id) = self.pin_id.take() {
2304 let mut state = self.trunk.state.lock().expect("Trunk state lock poisoned");
2305 state.segments.pins.remove(&pin_id);
2306 drop(state);
2307 self.trunk.segment_pin_released.notify_all();
2308 }
2309 }
2310}
2311
2312/// One item [`EventCursor::poll`] can hand back: one event-log entry (which
2313/// may itself still be `Segment`/`Utc`-anchored — a cursor sees an entry
2314/// the instant it is published, not only once it resolves; see
2315/// [`EventEntry::anchor`]), or a loss report.
2316///
2317/// `#[non_exhaustive]`: the growth point for anything a cursor might need
2318/// to surface beyond "entry" or "loss" later, without a breaking change to
2319/// every match arm in the workspace.
2320#[derive(Debug, Clone)]
2321#[non_exhaustive]
2322pub enum EventCursorItem {
2323 /// One event-log entry, in publish order.
2324 Event(EventEntry),
2325 /// This cursor fell behind the event log's ordinary
2326 /// [`TrunkConfig::event_capacity`] eviction: `skipped` entries were
2327 /// evicted before it read them. Exactly [`SampleCursorItem::Lagged`]'s
2328 /// contract.
2329 Lagged {
2330 /// Exact count of entries evicted since this cursor's last
2331 /// successful read.
2332 skipped: u64,
2333 },
2334}
2335
2336/// A subscribed reader of a [`Trunk`]'s event log. Obtained via
2337/// [`Trunk::subscribe_events`] — read that method's docs, and
2338/// [`Trunk::subscribe`]'s fan-out guidance, before creating more than a
2339/// handful of these.
2340pub struct EventCursor {
2341 trunk: Arc<Trunk>,
2342 consumed: u64,
2343}
2344
2345impl EventCursor {
2346 /// Pull the next item, if any is ready. Loss is always reported before
2347 /// further data — the same cannot-be-skipped-past precedent as
2348 /// [`SampleCursor::poll`]/[`SegmentCursor::poll`].
2349 pub fn poll(&mut self) -> Option<EventCursorItem> {
2350 let state = self.trunk.state.lock().expect("Trunk state lock poisoned");
2351 let log = &state.events;
2352 if self.consumed < log.base {
2353 let skipped = log.base - self.consumed;
2354 self.consumed = log.base;
2355 return Some(EventCursorItem::Lagged { skipped });
2356 }
2357 let idx = (self.consumed - log.base) as usize;
2358 if let Some(entry) = log.entries.get(idx) {
2359 self.consumed += 1;
2360 return Some(EventCursorItem::Event(entry.clone()));
2361 }
2362 None
2363 }
2364}
2365
2366#[cfg(test)]
2367mod tests {
2368 use super::*;
2369 use std::sync::mpsc;
2370 use std::thread;
2371 use transmux::pipeline::{CodecConfig, DataCarriage};
2372
2373 /// An opaque `TrackSpec` for track-set tests — mirrors `ingress`'s own
2374 /// identically-named test helper (same shape, so a track built here and
2375 /// one built there compare equal field-for-field for any given
2376 /// `track_id`).
2377 fn opaque_track(track_id: u32) -> TrackSpec {
2378 TrackSpec::new(
2379 track_id,
2380 90_000,
2381 CodecConfig::Data {
2382 stream_type: 0x06,
2383 descriptors: Vec::new(),
2384 carriage: DataCarriage::Pes,
2385 },
2386 )
2387 }
2388
2389 /// `NonZeroUsize` from a literal capacity, for readability at the ~30
2390 /// `TrunkConfig::new` call sites below. Panicking on `0` here is correct
2391 /// and is *not* the behaviour the deleted `zero_*_capacity_panics` tests
2392 /// asserted: this is a test helper rejecting a typo in test source, not
2393 /// the library accepting then rejecting a zero at run time — the library
2394 /// can no longer be handed one at all.
2395 fn nz(n: usize) -> NonZeroUsize {
2396 NonZeroUsize::new(n).expect("test capacity must be non-zero")
2397 }
2398
2399 fn sample(byte: u8, len: usize) -> Sample {
2400 Sample::new(Bytes::from(vec![byte; len]), Some(0), Some(0), None, true)
2401 }
2402
2403 fn timed_data(item: &SampleCursorItem) -> Option<(u32, &Sample)> {
2404 match item {
2405 SampleCursorItem::Timed { track_id, sample } => Some((*track_id, sample)),
2406 _ => None,
2407 }
2408 }
2409
2410 fn segment_entry(byte: u8, seq: u32) -> SegmentEntry {
2411 SegmentEntry::new(
2412 Bytes::from(vec![byte; 16]),
2413 seq,
2414 Duration::from_secs(2),
2415 Timestamp::from_nanos(u64::from(seq) * 2_000_000_000),
2416 SegmentMeta {
2417 discontinuous: false,
2418 },
2419 )
2420 }
2421
2422 fn segment_data(item: &SegmentCursorItem) -> Option<&SegmentEntry> {
2423 match item {
2424 SegmentCursorItem::Segment(entry) => Some(entry),
2425 _ => None,
2426 }
2427 }
2428
2429 /// Drains up to `n` items from `cursor`, stopping early if `poll`
2430 /// returns `None` — see [`drain`]'s doc for why this is bounded rather
2431 /// than looping until `None`.
2432 fn drain_segments(cursor: &mut SegmentCursor, n: usize) -> Vec<SegmentCursorItem> {
2433 let mut out = Vec::new();
2434 for _ in 0..n {
2435 match cursor.poll() {
2436 Some(item) => out.push(item),
2437 None => break,
2438 }
2439 }
2440 out
2441 }
2442
2443 /// Drains up to `n` items from `cursor`, stopping early if `poll`
2444 /// returns `None` — a bounded collection loop so a mutation that never
2445 /// advances `*_consumed` (and would otherwise re-yield the same item
2446 /// forever) fails the test's length/content assertions instead of
2447 /// hanging it.
2448 fn drain(cursor: &mut SampleCursor, n: usize) -> Vec<SampleCursorItem> {
2449 let mut out = Vec::new();
2450 for _ in 0..n {
2451 match cursor.poll() {
2452 Some(item) => out.push(item),
2453 None => break,
2454 }
2455 }
2456 out
2457 }
2458
2459 // --- 1. multiple cursors, every sample, in order, no dup/no loss -----
2460
2461 /// MUTATION VERIFIED: removing `self.timed_consumed += 1;` from the
2462 /// `Timed`-data return arm of `SampleCursor::poll` (so the same ring
2463 /// index is re-read every call) makes this test fail — `drain` still
2464 /// returns exactly 5 items (poll never runs out), but they are five
2465 /// copies of the first published sample (`byte = 0`) instead of the
2466 /// distinct sequence `0..5`, so the `assert_eq!` on the reconstructed
2467 /// byte sequence fails with a mismatch at index 1. Recompiled and
2468 /// re-run to confirm the failure, then reverted.
2469 #[test]
2470 fn multiple_cursors_see_every_sample_in_order_with_no_dup_or_loss() {
2471 let trunk = Trunk::new(TrunkConfig::new(nz(100), nz(10), nz(4), nz(8), nz(8)));
2472 let mut c1 = trunk.subscribe();
2473 let mut c2 = trunk.subscribe();
2474 let mut c3 = trunk.subscribe();
2475 let writer = trunk.writer().unwrap();
2476
2477 for i in 0u8..5 {
2478 writer.publish(7, RetentionClass::Timed, sample(i, 16));
2479 }
2480
2481 for cursor in [&mut c1, &mut c2, &mut c3] {
2482 let items = drain(cursor, 5);
2483 assert_eq!(items.len(), 5, "each cursor must see exactly 5 samples");
2484 let bytes: Vec<u8> = items
2485 .iter()
2486 .map(|item| timed_data(item).unwrap().1.data[0])
2487 .collect();
2488 assert_eq!(bytes, vec![0, 1, 2, 3, 4], "must be in publish order");
2489 assert!(cursor.poll().is_none(), "no extra/duplicated items");
2490 }
2491 }
2492
2493 // --- 2. slow reader lags, writer completes regardless -----------------
2494
2495 /// MUTATION VERIFIED: changing `ClassLog::push`'s eviction condition from
2496 /// `self.entries.len() == self.capacity` to `false` (i.e. disabling
2497 /// eviction, simulating a writer that would instead have to wait/reject
2498 /// once "full") makes `trunk.timed_len()` grow to 1024 instead of
2499 /// staying at the configured cap of 4, and the lag report's `skipped`
2500 /// reads back as `0` (base never advances), not `1020`. Recompiled and
2501 /// re-run to confirm the failure, then reverted.
2502 #[test]
2503 fn slow_reader_lags_but_writer_completes_regardless() {
2504 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(10), nz(4), nz(8), nz(8)));
2505 let mut slow = trunk.subscribe();
2506 let writer = trunk.writer().unwrap();
2507
2508 // The slow reader never polls while 1024 samples are published —
2509 // there is no wait-for-reader code path in `publish` for this loop
2510 // to block on (see `TrunkWriter`'s docs), so this simply completes.
2511 // A single thread is sufficient to demonstrate this: the absence of
2512 // a blocking path is a structural property of `publish`, not a race
2513 // that needs real concurrency to expose (`crate::byte_tap`'s
2514 // equivalent test uses the same reasoning).
2515 for i in 0u8..=255u8 {
2516 for _ in 0..4 {
2517 writer.publish(1, RetentionClass::Timed, sample(i, 8));
2518 }
2519 }
2520 // 256 * 4 = 1024 published; ring capacity is 4.
2521 assert_eq!(
2522 trunk.timed_len(),
2523 4,
2524 "writer unblocked: ring stayed bounded"
2525 );
2526
2527 let first = slow.poll().unwrap();
2528 assert!(
2529 matches!(first, SampleCursorItem::Lagged { skipped: 1020 }),
2530 "expected Lagged{{skipped: 1020}}, got {first:?}"
2531 );
2532 }
2533
2534 // --- 3. lag reports an accurate skipped count -------------------------
2535
2536 /// MUTATION VERIFIED: changing the `skipped` computation in
2537 /// `SampleCursor::poll`'s `Timed`-lag branch from
2538 /// `state.timed.base - self.timed_consumed` to
2539 /// `state.timed.base - self.timed_consumed + 1` makes this test fail:
2540 /// expected `skipped: 6`, got `skipped: 7`. Recompiled and re-run to
2541 /// confirm the failure, then reverted.
2542 #[test]
2543 fn lag_is_reported_with_an_accurate_skipped_count() {
2544 let trunk = Trunk::new(TrunkConfig::new(nz(3), nz(10), nz(4), nz(8), nz(8)));
2545 let mut cursor = trunk.subscribe();
2546 let writer = trunk.writer().unwrap();
2547
2548 // Capacity 3, publish 9: 6 evicted before the cursor ever reads.
2549 for i in 0u8..9 {
2550 writer.publish(2, RetentionClass::Timed, sample(i, 4));
2551 }
2552
2553 let first = cursor.poll().unwrap();
2554 assert!(
2555 matches!(first, SampleCursorItem::Lagged { skipped: 6 }),
2556 "expected Lagged{{skipped: 6}}, got {first:?}"
2557 );
2558
2559 // The remaining 3 (bytes 6,7,8) must still be readable, in order.
2560 let items = drain(&mut cursor, 3);
2561 let bytes: Vec<u8> = items
2562 .iter()
2563 .map(|item| timed_data(item).unwrap().1.data[0])
2564 .collect();
2565 assert_eq!(bytes, vec![6, 7, 8]);
2566 assert!(cursor.poll().is_none());
2567 }
2568
2569 // --- 4. Sparse loss reports Degraded, distinct from Timed's Lagged ----
2570
2571 /// MUTATION VERIFIED: changing the `Sparse`-lag branch of
2572 /// `SampleCursor::poll` to also return `SampleCursorItem::Lagged` (i.e.
2573 /// collapsing the two variants) makes the
2574 /// `matches!(item, SampleCursorItem::Degraded { .. })` assertion below
2575 /// fail — the item is a `Lagged` instead. Recompiled and re-run to
2576 /// confirm the failure, then reverted.
2577 #[test]
2578 fn sparse_reader_loses_data_reports_degraded_distinguishable_from_timed_lagged() {
2579 let trunk = Trunk::new(TrunkConfig::new(nz(2), nz(2), nz(4), nz(8), nz(8)));
2580 let mut cursor = trunk.subscribe();
2581 let writer = trunk.writer().unwrap();
2582
2583 // Overflow the Timed ring (cap 2) with 5 publishes: ordinary loss.
2584 for i in 0u8..5 {
2585 writer.publish(3, RetentionClass::Timed, sample(i, 4));
2586 }
2587 // Overflow the Sparse ring (cap 2) with 4 publishes: escalated loss.
2588 for i in 0u8..4 {
2589 writer.publish(9, RetentionClass::Sparse, sample(100 + i, 4));
2590 }
2591
2592 let timed_loss = cursor.poll().unwrap();
2593 assert!(
2594 matches!(timed_loss, SampleCursorItem::Lagged { skipped: 3 }),
2595 "expected ordinary Lagged{{skipped: 3}} for the Timed ring, got {timed_loss:?}"
2596 );
2597
2598 let sparse_loss = cursor.poll().unwrap();
2599 assert!(
2600 matches!(sparse_loss, SampleCursorItem::Degraded { skipped: 2 }),
2601 "expected escalated Degraded{{skipped: 2}} for the Sparse ring, got {sparse_loss:?}"
2602 );
2603 assert_ne!(
2604 core::mem::discriminant(&timed_loss),
2605 core::mem::discriminant(&sparse_loss),
2606 "Lagged and Degraded must be distinct variants, not merely different field values"
2607 );
2608 }
2609
2610 // --- 5. the ring is bounded: flooding cannot grow memory unboundedly --
2611
2612 /// MUTATION VERIFIED: removing the eviction check in `ClassLog::push`
2613 /// (replacing `if self.entries.len() == self.capacity { .. }` with a
2614 /// no-op) makes `trunk.timed_len()`/`trunk.sparse_len()` grow well past
2615 /// the configured caps (`4`/`3`) instead of staying bounded — the
2616 /// assertions inside the flood loop below fail on the first
2617 /// over-capacity iteration. Recompiled and re-run to confirm the
2618 /// failure, then reverted.
2619 #[test]
2620 fn ring_is_bounded_under_flood_on_both_classes() {
2621 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(3), nz(4), nz(8), nz(8)));
2622 let writer = trunk.writer().unwrap();
2623
2624 for i in 0u32..50_000 {
2625 writer.publish(5, RetentionClass::Timed, sample((i % 256) as u8, 2));
2626 assert!(
2627 trunk.timed_len() <= 4,
2628 "Timed ring exceeded its cap mid-flood"
2629 );
2630 if i % 7 == 0 {
2631 writer.publish(6, RetentionClass::Sparse, sample((i % 256) as u8, 2));
2632 assert!(
2633 trunk.sparse_len() <= 3,
2634 "Sparse ring exceeded its cap mid-flood"
2635 );
2636 }
2637 }
2638 assert_eq!(trunk.timed_len(), 4);
2639 assert_eq!(trunk.sparse_len(), 3);
2640 }
2641
2642 // --- 5b. subscribe_from_backlog: exact replay + Lagged on overwrite ---
2643
2644 /// [`Trunk::subscribe_from_backlog`]'s core promise: a cursor subscribed
2645 /// *after* samples have already landed in both rings still sees them,
2646 /// exactly (in order, no dup), for each retention class independently —
2647 /// the property issue #808's `ProgramSegmenter` fix depends on.
2648 ///
2649 /// MUTATION VERIFIED: changing `subscribe_from_backlog`'s
2650 /// `timed_consumed: state.timed.base` to
2651 /// `timed_consumed: state.timed.published` (i.e. accidentally reusing
2652 /// `subscribe`'s live-tail initialisation) makes this test's
2653 /// `assert_eq!(timed_bytes, vec![10, 11, 12])` fail: `drain` returns an
2654 /// empty `Vec` (actual) instead of the expected `[10, 11, 12]`, because
2655 /// `timed_consumed` now equals `published` — every published entry
2656 /// already counts as "consumed" the instant the cursor is created, so
2657 /// `poll` immediately returns `None` instead of replaying the resident
2658 /// backlog. Recompiled and re-run to confirm this exact failure, then
2659 /// reverted.
2660 #[test]
2661 fn subscribe_from_backlog_replays_exact_resident_entries_both_classes() {
2662 let trunk = Trunk::new(TrunkConfig::new(nz(100), nz(100), nz(4), nz(8), nz(8)));
2663 let writer = trunk.writer().unwrap();
2664
2665 // Published *before* the cursor exists — subscribe() would never see
2666 // any of this; subscribe_from_backlog() must replay all of it, since
2667 // ring capacity (100) is nowhere near exhausted.
2668 for i in 10u8..13 {
2669 writer.publish(1, RetentionClass::Timed, sample(i, 4));
2670 }
2671 for i in 20u8..22 {
2672 writer.publish(2, RetentionClass::Sparse, sample(i, 4));
2673 }
2674
2675 let mut cursor = trunk.subscribe_from_backlog();
2676
2677 // Merge order (module docs): pending lag reports first (none here),
2678 // then Sparse data, then Timed data.
2679 let sparse_items = drain(&mut cursor, 2);
2680 let sparse_bytes: Vec<u8> = sparse_items
2681 .iter()
2682 .map(|item| match item {
2683 SampleCursorItem::Sparse { sample, .. } => sample.data[0],
2684 other => panic!("expected Sparse, got {other:?}"),
2685 })
2686 .collect();
2687 assert_eq!(sparse_bytes, vec![20, 21], "exact resident Sparse backlog");
2688
2689 let timed_items = drain(&mut cursor, 3);
2690 let timed_bytes: Vec<u8> = timed_items
2691 .iter()
2692 .map(|item| timed_data(item).unwrap().1.data[0])
2693 .collect();
2694 assert_eq!(
2695 timed_bytes,
2696 vec![10, 11, 12],
2697 "exact resident Timed backlog"
2698 );
2699
2700 assert!(
2701 cursor.poll().is_none(),
2702 "no extra items beyond the resident backlog"
2703 );
2704
2705 // New samples published after subscribing still flow through
2706 // normally, proving this cursor is a real live cursor afterwards,
2707 // not a one-shot snapshot.
2708 writer.publish(1, RetentionClass::Timed, sample(99, 4));
2709 let live = cursor.poll().unwrap();
2710 assert_eq!(timed_data(&live).unwrap().1.data[0], 99);
2711 }
2712
2713 /// When the backlog a [`Trunk::subscribe_from_backlog`] cursor would
2714 /// have replayed has *already* been evicted by ring capacity before the
2715 /// subscribe call, this cursor must behave exactly like an ordinary
2716 /// [`Trunk::subscribe`] cursor from that point on: report the loss
2717 /// in-band as `Lagged`/`Degraded`, never silently skip it.
2718 ///
2719 /// MUTATION VERIFIED: changing `subscribe_from_backlog`'s
2720 /// `timed_consumed: state.timed.base` to `timed_consumed: 0` (simulating
2721 /// "replay from the beginning of time" rather than "replay what the ring
2722 /// still holds", anchoring at the wrong reference point) makes this
2723 /// test's `assert!(matches!(first, SampleCursorItem::Lagged { skipped: 6
2724 /// }))` fail: actual `Lagged { skipped: 12 }` (`12 - 0`, counting the 6
2725 /// entries already evicted *before* this cursor even subscribed as if
2726 /// they were its own loss) instead of the expected `Lagged { skipped: 6
2727 /// }` (`12 - 6`, only the 6 evictions that happened *after* this cursor
2728 /// subscribed). Recompiled and re-run to confirm this exact failure,
2729 /// then reverted.
2730 #[test]
2731 fn subscribe_from_backlog_reports_lagged_when_backlog_already_overwritten() {
2732 let trunk = Trunk::new(TrunkConfig::new(nz(3), nz(10), nz(4), nz(8), nz(8)));
2733 let writer = trunk.writer().unwrap();
2734
2735 // Capacity 3, publish 9: only bytes 6,7,8 remain resident; 0..6 are
2736 // already gone by the time subscribe_from_backlog is called — this
2737 // cursor must NOT report those 6 as its own loss (they were never
2738 // its backlog to miss), which is exactly why it anchors at `base`
2739 // (6), not `0`.
2740 for i in 0u8..9 {
2741 writer.publish(1, RetentionClass::Timed, sample(i, 4));
2742 }
2743 let mut cursor = trunk.subscribe_from_backlog();
2744
2745 // Publish 6 more before this cursor ever polls — the ring is
2746 // already full (capacity 3), so each push evicts exactly one
2747 // resident entry, advancing `base` from 6 to 12. This IS loss this
2748 // cursor is responsible for (it subscribed to a live cursor at 6,
2749 // then fell behind by 6 before its first poll).
2750 for i in 9u8..15 {
2751 writer.publish(1, RetentionClass::Timed, sample(i, 4));
2752 }
2753
2754 let first = cursor.poll().unwrap();
2755 assert!(
2756 matches!(first, SampleCursorItem::Lagged { skipped: 6 }),
2757 "expected Lagged{{skipped: 6}}, got {first:?}"
2758 );
2759
2760 // The remaining resident 3 (bytes 12,13,14) must still be readable.
2761 let items = drain(&mut cursor, 3);
2762 let bytes: Vec<u8> = items
2763 .iter()
2764 .map(|item| timed_data(item).unwrap().1.data[0])
2765 .collect();
2766 assert_eq!(bytes, vec![12, 13, 14]);
2767 assert!(cursor.poll().is_none());
2768 }
2769
2770 // --- 6. payload sharing: Bytes::as_ptr() identity, not equality -------
2771
2772 /// MUTATION VERIFIED: replacing `sample.clone()` in both of
2773 /// `SampleCursor::poll`'s data-return arms with a hand-rolled copy
2774 /// (`Sample::new(Bytes::copy_from_slice(sample.data.as_ref()), ...)`,
2775 /// preserving every field's *value* so content-equality would still
2776 /// hold) makes this test's pointer-identity assertion fail — with the
2777 /// mutation, `p1 == p2` is `false` (two distinct heap allocations with
2778 /// equal contents), whereas the unmutated `clone()` path yields
2779 /// `p1 == p2 == p3`. This is precisely the distinction a
2780 /// content-equality assertion would have missed. Recompiled and re-run
2781 /// to confirm the failure, then reverted.
2782 #[test]
2783 fn payload_is_shared_not_copied_across_cursors() {
2784 let trunk = Trunk::new(TrunkConfig::new(nz(8), nz(8), nz(4), nz(8), nz(8)));
2785 let mut c1 = trunk.subscribe();
2786 let mut c2 = trunk.subscribe();
2787 let mut c3 = trunk.subscribe();
2788 let writer = trunk.writer().unwrap();
2789
2790 writer.publish(4, RetentionClass::Timed, sample(0xAB, 65536));
2791
2792 let i1 = c1.poll().unwrap();
2793 let i2 = c2.poll().unwrap();
2794 let i3 = c3.poll().unwrap();
2795 let p1 = timed_data(&i1).unwrap().1.data.as_ptr();
2796 let p2 = timed_data(&i2).unwrap().1.data.as_ptr();
2797 let p3 = timed_data(&i3).unwrap().1.data.as_ptr();
2798
2799 assert_eq!(
2800 p1, p2,
2801 "cursor 2's payload must be the SAME allocation as cursor 1's"
2802 );
2803 assert_eq!(
2804 p2, p3,
2805 "cursor 3's payload must be the SAME allocation as cursor 1's"
2806 );
2807 // Not just equal contents (that would also pass for two independent
2808 // 64KiB copies) — the ptr comparison above is the real assertion;
2809 // this just confirms the payload wasn't corrupted in the process.
2810 assert_eq!(timed_data(&i1).unwrap().1.data.len(), 65536);
2811 }
2812
2813 // --- Construction invariants -------------------------------------------
2814 //
2815 // The five `zero_*_capacity_panics` tests that used to live here are
2816 // deliberately GONE, not merely disabled: every `TrunkConfig` capacity is
2817 // now a `NonZeroUsize`, so a zero capacity is unrepresentable rather than
2818 // rejected at run time, and `Trunk::new` no longer has (or needs) the
2819 // `assert!`s they pinned. A test asserting a panic that can no longer
2820 // occur would not compile against the new signature anyway, and keeping a
2821 // rewritten version would only be asserting that `NonZeroUsize::new(0)`
2822 // returns `None` — a property of the standard library, not of this crate.
2823
2824 #[test]
2825 fn second_writer_is_refused() {
2826 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(8)));
2827 let _first = trunk.writer().unwrap();
2828 assert!(
2829 trunk.writer().is_none(),
2830 "a Trunk has exactly one sample/event writer"
2831 );
2832 }
2833
2834 // --- W1. the SegmentWriter half of the split is single-take too, ------
2835 // --- independently of TrunkWriter ---------------------------------------
2836
2837 /// MUTATION VERIFIED: replacing `Trunk::segment_writer`'s
2838 /// `compare_exchange` call with an unconditional `Some(SegmentWriter {
2839 /// .. })` (i.e. reintroducing "anyone can take it, any number of times")
2840 /// makes this test's `assert!(trunk.segment_writer().is_none(), ..)`
2841 /// fail — the second call succeeds instead of being refused. Recompiled
2842 /// and re-run to confirm the failure, then reverted.
2843 #[test]
2844 fn second_segment_writer_is_refused() {
2845 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(8)));
2846 let _first = trunk.segment_writer().unwrap();
2847 assert!(
2848 trunk.segment_writer().is_none(),
2849 "a Trunk has exactly one segment/part writer"
2850 );
2851 }
2852
2853 // --- W2. THE GAP THIS STEP CLOSES: a sample/event writer and a --------
2854 // --- segment/part writer can be held AT THE SAME TIME -------------------
2855
2856 /// This is the property that was **structurally impossible** before this
2857 /// step: `Trunk::writer()` and a hypothetical segment-writing capability
2858 /// shared one `AtomicBool`, so whichever component (the ingest driver)
2859 /// took the one writer made it impossible for anything else (a
2860 /// segmenter) to ever publish a segment or a part.
2861 ///
2862 /// MUTATION VERIFIED: changing `Trunk::segment_writer` to gate on
2863 /// `self.writer_taken` instead of its own `self.segment_writer_taken`
2864 /// (i.e. reintroducing the single-shared-flag bug this step fixes) makes
2865 /// this test's `let segments = trunk.segment_writer().unwrap();` line
2866 /// panic — `segment_writer()` returns `None` because the sample/event
2867 /// writer taken just above already flipped the shared flag. Recompiled
2868 /// and re-run to confirm the failure, then reverted.
2869 #[test]
2870 fn sample_and_segment_writers_can_be_held_simultaneously() {
2871 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(8)));
2872 let samples = trunk.writer().unwrap();
2873 let segments = trunk.segment_writer().expect(
2874 "the segment/part writer must still be takeable while the sample/event writer is held",
2875 );
2876
2877 // Both are simultaneously live and independently usable — not merely
2878 // both `Some` a moment apart.
2879 samples.publish(1, RetentionClass::Timed, sample(1, 4));
2880 segments.publish_segment(segment_entry(1, 1));
2881 assert_eq!(trunk.timed_len(), 1);
2882 assert_eq!(trunk.segment_len(), 1);
2883 }
2884
2885 // --- W3. THE SEGMENTER-SHAPED, END-TO-END PROPERTY: a segmenter reads --
2886 // --- samples through its own cursor and publishes the segment/part it --
2887 // --- derives from them through the OTHER writer — unreachable before ---
2888 // --- this step, since there was only one writer for the whole Trunk ----
2889
2890 /// The load-bearing test for this step: models exactly the component the
2891 /// gap analysis found could not exist — a segmenter that holds a
2892 /// [`SampleCursor`] (to read the samples it segments) *and* a
2893 /// [`SegmentWriter`] (to publish what it produces) at the same time,
2894 /// distinct from the ingest driver's own [`TrunkWriter`].
2895 ///
2896 /// MUTATION VERIFIED: commenting out `state.segments.push(entry);` in
2897 /// `SegmentWriter::publish_segment` (simulating "the moved method is a
2898 /// stub that does not actually reach the ring") makes this test's
2899 /// `let got = seg_cursor.poll().expect(..)` panic — nothing was ever
2900 /// pushed, so the segment log stays empty and the cursor has nothing to
2901 /// return. Recompiled and re-run to confirm the failure, then reverted.
2902 #[test]
2903 fn segmenter_holds_sample_cursor_and_segment_writer_at_once() {
2904 let trunk = Trunk::new(TrunkConfig::new(nz(8), nz(4), nz(4), nz(8), nz(8)));
2905
2906 // The ingest driver's own handle — a different component, a
2907 // different ring group.
2908 let ingest = trunk.writer().unwrap();
2909 // The segmenter's read side (its own SampleCursor) and write side
2910 // (the SegmentWriter) — held together, which is exactly what the
2911 // single-writer-per-Trunk model made impossible.
2912 let mut samples = trunk.subscribe();
2913 let segmenter = trunk.segment_writer().expect(
2914 "a segmenter must be able to take the segment/part writer \
2915 while the ingest driver still holds the sample/event writer",
2916 );
2917 // `subscribe_segments` only sees entries published *after* this call
2918 // (exactly `Trunk::subscribe`'s "starts from now" contract) — taken
2919 // up front so the read-back below has something to see.
2920 let mut seg_cursor = trunk.subscribe_segments();
2921
2922 for i in 0u8..3 {
2923 ingest.publish(1, RetentionClass::Timed, sample(i, 4));
2924 }
2925
2926 // The segmenter consumes exactly the samples it is about to derive
2927 // a segment from.
2928 let mut muxed = Vec::new();
2929 for _ in 0..3 {
2930 match samples.poll() {
2931 Some(SampleCursorItem::Timed { sample, .. }) => {
2932 muxed.push(sample.data[0]);
2933 }
2934 other => panic!("expected a Timed sample, got {other:?}"),
2935 }
2936 }
2937 assert_eq!(muxed, vec![0, 1, 2]);
2938
2939 // ...then publishes the segment (and one live part of it) derived
2940 // from exactly those samples — through the OTHER writer.
2941 segmenter.publish_part(part_entry(0xAB, 1, 0));
2942 segmenter.publish_segment(segment_entry(0xAA, 1));
2943
2944 // Read both back through the segment/part log's own query surface —
2945 // proving the publish actually reached the shared Trunk, not just a
2946 // private buffer inside the segmenter.
2947 let got = seg_cursor
2948 .poll()
2949 .expect("the segmenter's published segment must be visible");
2950 assert_eq!(segment_data(&got).unwrap().sequence_number, 1);
2951 assert_eq!(
2952 trunk.part_bytes(1, 0),
2953 Some(Bytes::from(vec![0xAB; 8])),
2954 "the segmenter's published part must be individually addressable too"
2955 );
2956 }
2957
2958 #[test]
2959 fn subscribe_starts_from_now_not_from_history() {
2960 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(8)));
2961 let writer = trunk.writer().unwrap();
2962 writer.publish(1, RetentionClass::Timed, sample(1, 4));
2963 writer.publish(1, RetentionClass::Timed, sample(2, 4));
2964
2965 // Subscribing after two publishes must not see either of them.
2966 let mut cursor = trunk.subscribe();
2967 assert!(cursor.poll().is_none());
2968
2969 writer.publish(1, RetentionClass::Timed, sample(3, 4));
2970 let item = cursor.poll().unwrap();
2971 assert_eq!(timed_data(&item).unwrap().1.data[0], 3);
2972 }
2973
2974 // ===================== segment log ====================================
2975
2976 // --- S1. multiple cursors, every segment, in order, no dup/no loss ----
2977
2978 /// MUTATION VERIFIED: removing the `self.consumed += 1;` from the
2979 /// non-pinning data-return arm of `SegmentCursor::poll` (so the same
2980 /// ring index is re-read every call) makes this test fail exactly like
2981 /// `SampleCursor::poll`'s equivalent mutation: `drain_segments` still
2982 /// returns 5 items, but all 5 are the first published segment
2983 /// (`sequence_number == 1`) instead of the distinct sequence `1..=5`, so
2984 /// the `assert_eq!` on the reconstructed sequence-number list fails at
2985 /// index 1. Recompiled and re-run to confirm the failure, then reverted.
2986 #[test]
2987 fn multiple_segment_cursors_see_every_segment_in_order_with_no_dup_or_loss() {
2988 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(100), nz(8), nz(8)));
2989 let mut c1 = trunk.subscribe_segments();
2990 let mut c2 = trunk.subscribe_segments();
2991 let mut c3 = trunk.subscribe_segments();
2992 let writer = trunk.segment_writer().unwrap();
2993
2994 for i in 0u32..5 {
2995 writer.publish_segment(segment_entry(i as u8, i + 1));
2996 }
2997
2998 for cursor in [&mut c1, &mut c2, &mut c3] {
2999 let items = drain_segments(cursor, 5);
3000 assert_eq!(items.len(), 5, "each cursor must see exactly 5 segments");
3001 let seqs: Vec<u32> = items
3002 .iter()
3003 .map(|item| segment_data(item).unwrap().sequence_number)
3004 .collect();
3005 assert_eq!(seqs, vec![1, 2, 3, 4, 5], "must be in playlist order");
3006 assert!(cursor.poll().is_none(), "no extra/duplicated items");
3007 }
3008 }
3009
3010 // --- S2. a non-pinning slow reader lags; writer completes regardless --
3011
3012 /// MUTATION VERIFIED: changing `SegmentLog::push`'s eviction condition
3013 /// from `self.entries.len() == self.capacity` to `false` (disabling
3014 /// eviction) makes `trunk.segment_len()` grow to 1024 instead of staying
3015 /// at the configured cap of 4, and the subsequent `Lagged` assertion
3016 /// fails because `base` never advanced (`skipped` reads back as `0`, not
3017 /// `1020`). Recompiled and re-run to confirm the failure, then reverted.
3018 #[test]
3019 fn non_pinning_slow_segment_reader_lags_but_writer_completes_regardless() {
3020 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(8)));
3021 let mut slow = trunk.subscribe_segments();
3022 let writer = trunk.segment_writer().unwrap();
3023
3024 // The slow (non-pinning) reader never polls while 1024 segments are
3025 // published — there is no wait-for-reader path for a non-pinning
3026 // cursor, so this simply completes (same reasoning as
3027 // `slow_reader_lags_but_writer_completes_regardless`).
3028 for i in 0u32..1024 {
3029 writer.publish_segment(segment_entry((i % 256) as u8, i + 1));
3030 }
3031 assert_eq!(
3032 trunk.segment_len(),
3033 4,
3034 "writer unblocked: segment log stayed bounded"
3035 );
3036
3037 let first = slow.poll().unwrap();
3038 assert!(
3039 matches!(first, SegmentCursorItem::Lagged { skipped: 1020 }),
3040 "expected Lagged{{skipped: 1020}}, got {first:?}"
3041 );
3042 }
3043
3044 // --- S3. THE DVR PROPERTY: a pinning reader loses nothing while a ------
3045 // --- non-pinning sibling lags, and StallIngest is what makes it true --
3046
3047 /// MUTATION VERIFIED: changing the `must_wait` computation in
3048 /// `SegmentWriter::publish_segment`'s `ArchiveOverrun::StallIngest` arm
3049 /// from `must_wait = true;` to `{}` (a no-op, i.e. treating
3050 /// `StallIngest` exactly like `Gap`) makes this test fail: the third
3051 /// `publish_segment` call no longer blocks, so the background-thread
3052 /// completion channel's `recv_timeout` at the "still blocked" checkpoint
3053 /// returns `Ok(())` instead of timing out, and the assertion that it
3054 /// timed out (`is_err()`) fails. Recompiled and re-run to confirm the
3055 /// failure, then reverted.
3056 #[test]
3057 fn pinning_reader_receives_every_segment_while_non_pinning_reader_lags() {
3058 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(2), nz(8), nz(8)));
3059 let mut slow = trunk.subscribe_segments(); // non-pinning: will lag
3060 let mut archive = trunk.pin_segments(ArchiveOverrun::StallIngest); // pinning: must lose nothing
3061 let writer = Arc::new(trunk.segment_writer().unwrap());
3062
3063 // Fill the segment log's capacity (2) without any eviction yet.
3064 writer.publish_segment(segment_entry(1, 1));
3065 writer.publish_segment(segment_entry(2, 2));
3066
3067 // A third publish must evict the oldest (seq 1), which `archive`'s
3068 // pin has not yet consumed — with `StallIngest`, this call blocks.
3069 // Run it on a background thread (the same one `SegmentWriter`,
3070 // shared via `Arc` — this is the segment/part ring group's own
3071 // single-writer invariant, just called from a different thread) and
3072 // prove, via a completion channel, that it has NOT returned yet.
3073 let (done_tx, done_rx) = mpsc::channel();
3074 let blocked_writer = Arc::clone(&writer);
3075 let handle = thread::spawn(move || {
3076 blocked_writer.publish_segment(segment_entry(3, 3));
3077 done_tx.send(()).unwrap();
3078 });
3079
3080 assert!(
3081 done_rx.recv_timeout(Duration::from_millis(200)).is_err(),
3082 "publish_segment must still be blocked: archive has not consumed seq 1 yet"
3083 );
3084
3085 // `archive` catches up: consuming seq 1 releases its pin on it,
3086 // which must wake and unblock the writer thread.
3087 let first = archive.poll().unwrap();
3088 assert_eq!(segment_data(&first).unwrap().sequence_number, 1);
3089
3090 // HANG GUARD (issue #807): generous on purpose, same reasoning as
3091 // `retention.rs`'s `archive_overrun_stall_ingest_blocks_writer_until_driver_advances`
3092 // (the sibling proof of this same mechanism) -- the claim is "the
3093 // writer unblocks once the pin is drained", not "within N seconds".
3094 // The unblock is observed across a thread boundary, so a tight bound
3095 // measures the machine's scheduler, not this code.
3096 done_rx
3097 .recv_timeout(Duration::from_secs(60))
3098 .expect("publish_segment must unblock once the pin advances");
3099 handle.join().unwrap();
3100
3101 // `archive` receives every remaining segment with ZERO loss — no
3102 // `Gap`, no `Lagged` — proving the DVR property: pinning protected
3103 // it from the eviction that just happened.
3104 let second = archive.poll().unwrap();
3105 assert_eq!(segment_data(&second).unwrap().sequence_number, 2);
3106 let third = archive.poll().unwrap();
3107 assert_eq!(segment_data(&third).unwrap().sequence_number, 3);
3108 assert!(archive.poll().is_none());
3109
3110 // Meanwhile `slow` (non-pinning, never polled) DID lag: exactly one
3111 // segment (seq 1) was evicted out from under it.
3112 let lag = slow.poll().unwrap();
3113 assert!(
3114 matches!(lag, SegmentCursorItem::Lagged { skipped: 1 }),
3115 "expected Lagged{{skipped: 1}}, got {lag:?}"
3116 );
3117 let remaining: Vec<u32> = drain_segments(&mut slow, 2)
3118 .iter()
3119 .map(|item| segment_data(item).unwrap().sequence_number)
3120 .collect();
3121 assert_eq!(remaining, vec![2, 3]);
3122 }
3123
3124 // --- S4. pinning is bounded: an un-acking consumer cannot grow --------
3125 // --- memory without limit ----------------------------------------------
3126
3127 /// MUTATION VERIFIED: removing the eviction check in `SegmentLog::push`
3128 /// (replacing `if self.entries.len() == self.capacity { .. }` with a
3129 /// no-op, exactly like the sample-ring equivalent mutation) makes
3130 /// `trunk.segment_len()` grow past the configured cap of `4` instead of
3131 /// staying bounded — the in-loop assertion below fails on the first
3132 /// over-capacity iteration. Recompiled and re-run to confirm the
3133 /// failure, then reverted.
3134 #[test]
3135 fn pinning_is_bounded_an_unacking_consumer_cannot_grow_memory_without_limit() {
3136 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(8)));
3137 // Default policy (`Gap`) pinning cursor that never polls at all —
3138 // the worst case for memory growth: a dead/wedged archive consumer.
3139 let _archive = trunk.pin_segments(ArchiveOverrun::default());
3140 let writer = trunk.segment_writer().unwrap();
3141
3142 for i in 0u32..50_000 {
3143 writer.publish_segment(segment_entry((i % 256) as u8, i + 1));
3144 assert!(
3145 trunk.segment_len() <= 4,
3146 "segment log exceeded its cap mid-flood despite an un-acking pinning cursor"
3147 );
3148 }
3149 assert_eq!(trunk.segment_len(), 4);
3150 }
3151
3152 // --- S5. ArchiveOverrun::Gap gaps and reports --------------------------
3153
3154 /// MUTATION VERIFIED: changing the pinning branch of `SegmentCursor::poll`
3155 /// to report `SegmentCursorItem::Lagged` instead of `SegmentCursorItem::Gap`
3156 /// (collapsing the two, mirroring the sample path's Timed/Sparse
3157 /// mutation) makes the `matches!(item, SegmentCursorItem::Gap { .. })`
3158 /// assertion below fail — the item is a `Lagged` instead. Recompiled and
3159 /// re-run to confirm the failure, then reverted.
3160 #[test]
3161 fn archive_overrun_gap_evicts_and_reports_gap() {
3162 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(2), nz(8), nz(8)));
3163 let mut archive = trunk.pin_segments(ArchiveOverrun::Gap);
3164 let writer = trunk.segment_writer().unwrap();
3165
3166 // Publish 5 segments into a capacity-2 log without archive ever
3167 // polling: with `Gap`, eviction proceeds unconditionally, so this
3168 // never blocks.
3169 for i in 0u32..5 {
3170 writer.publish_segment(segment_entry(i as u8, i + 1));
3171 }
3172 assert_eq!(trunk.segment_len(), 2);
3173
3174 let gap = archive.poll().unwrap();
3175 assert!(
3176 matches!(gap, SegmentCursorItem::Gap { skipped: 3 }),
3177 "expected Gap{{skipped: 3}}, got {gap:?}"
3178 );
3179 // The recording has a hole, but the stream survives: archive keeps
3180 // reading the segments that remain.
3181 let remaining: Vec<u32> = drain_segments(&mut archive, 2)
3182 .iter()
3183 .map(|item| segment_data(item).unwrap().sequence_number)
3184 .collect();
3185 assert_eq!(remaining, vec![4, 5]);
3186 assert!(archive.poll().is_none());
3187 }
3188
3189 // --- S6. ArchiveOverrun::StallIngest actually applies back-pressure ---
3190
3191 /// MUTATION VERIFIED: same mutation and same observed failure as
3192 /// `pinning_reader_receives_every_segment_while_non_pinning_reader_lags`'s
3193 /// doc comment (removing `must_wait = true;` from the `StallIngest`
3194 /// arm) — this test is the narrower, single-purpose proof that
3195 /// `publish_segment` genuinely blocks, isolated from the sibling-lag
3196 /// scenario. Recompiled and re-run to confirm the failure, then
3197 /// reverted.
3198 #[test]
3199 fn archive_overrun_stall_ingest_actually_blocks_the_writer() {
3200 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(1), nz(8), nz(8)));
3201 let mut archive = trunk.pin_segments(ArchiveOverrun::StallIngest);
3202 let writer = Arc::new(trunk.segment_writer().unwrap());
3203
3204 writer.publish_segment(segment_entry(1, 1)); // fills capacity-1 log
3205
3206 let (done_tx, done_rx) = mpsc::channel();
3207 let blocked_writer = Arc::clone(&writer);
3208 let handle = thread::spawn(move || {
3209 blocked_writer.publish_segment(segment_entry(2, 2));
3210 done_tx.send(()).unwrap();
3211 });
3212
3213 assert!(
3214 done_rx.recv_timeout(Duration::from_millis(200)).is_err(),
3215 "publish_segment must block: the pin has not consumed seq 1 yet"
3216 );
3217
3218 let first = archive.poll().unwrap();
3219 assert_eq!(segment_data(&first).unwrap().sequence_number, 1);
3220
3221 // HANG GUARD (issue #807): generous on purpose, same reasoning as the
3222 // sibling test above (`pinning_reader_receives_every_segment_while_non_pinning_reader_lags`)
3223 // -- claim is "unblocks once drained", not "within N seconds"; the
3224 // unblock crosses a thread boundary so a tight bound measures the
3225 // scheduler, not this code.
3226 done_rx
3227 .recv_timeout(Duration::from_secs(60))
3228 .expect("publish_segment must unblock once the pin advances");
3229 handle.join().unwrap();
3230 }
3231
3232 // --- S7. ArchiveOverrun::Terminate drops the cursor --------------------
3233
3234 /// MUTATION VERIFIED: changing `ArchiveOverrun::Terminate => pin.terminated
3235 /// = true,` in `SegmentWriter::publish_segment` to `ArchiveOverrun::Terminate
3236 /// => {}` (a no-op, treating `Terminate` exactly like `Gap`) makes this
3237 /// test fail: `archive.poll()` returns `Some(Gap { .. })` instead of
3238 /// `Some(Terminated)`, so the `matches!` assertion on `Terminated` fails.
3239 /// Recompiled and re-run to confirm the failure, then reverted.
3240 #[test]
3241 fn archive_overrun_terminate_drops_the_cursor() {
3242 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(2), nz(8), nz(8)));
3243 let mut archive = trunk.pin_segments(ArchiveOverrun::Terminate);
3244 let writer = trunk.segment_writer().unwrap();
3245
3246 // Publish past capacity without archive ever polling: `Terminate`
3247 // never blocks (like `Gap`), so this completes.
3248 for i in 0u32..5 {
3249 writer.publish_segment(segment_entry(i as u8, i + 1));
3250 }
3251 assert_eq!(trunk.segment_len(), 2, "writer unblocked despite Terminate");
3252
3253 let item = archive.poll().unwrap();
3254 assert!(
3255 matches!(item, SegmentCursorItem::Terminated),
3256 "expected Terminated, got {item:?}"
3257 );
3258 // The cursor is done: every poll after `Terminated` returns `None`,
3259 // never resuming as if nothing happened.
3260 assert!(archive.poll().is_none());
3261 assert!(archive.poll().is_none());
3262
3263 // The log itself is unaffected: publishing continues to work, and a
3264 // fresh cursor still sees ordinary segment log behaviour.
3265 writer.publish_segment(segment_entry(9, 6));
3266 assert_eq!(trunk.segment_len(), 2);
3267 }
3268
3269 // --- S8. segment bytes are shared, not copied, across cursors ---------
3270
3271 /// MUTATION VERIFIED: replacing `entry.clone()` in
3272 /// `SegmentCursor::poll`'s non-pinning data-return arm with a hand-rolled
3273 /// copy (`SegmentEntry { bytes: Bytes::copy_from_slice(entry.bytes.as_ref()),
3274 /// ..entry.clone() }`, preserving every field's *value*) makes this
3275 /// test's pointer-identity assertion fail — `p1 == p2` becomes `false`
3276 /// (two distinct heap allocations with equal contents) instead of the
3277 /// unmutated `clone()` path's `p1 == p2 == p3`. This is exactly the
3278 /// distinction a content-equality assertion would have missed.
3279 /// Recompiled and re-run to confirm the failure, then reverted.
3280 #[test]
3281 fn segment_bytes_are_shared_not_copied_across_cursors() {
3282 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(8), nz(8), nz(8)));
3283 let mut c1 = trunk.subscribe_segments();
3284 let mut c2 = trunk.subscribe_segments();
3285 let mut c3 = trunk.subscribe_segments();
3286 let writer = trunk.segment_writer().unwrap();
3287
3288 writer.publish_segment(SegmentEntry::new(
3289 Bytes::from(vec![0xCDu8; 65536]),
3290 1,
3291 Duration::from_secs(2),
3292 Timestamp::from_nanos(0),
3293 SegmentMeta {
3294 discontinuous: false,
3295 },
3296 ));
3297
3298 let i1 = c1.poll().unwrap();
3299 let i2 = c2.poll().unwrap();
3300 let i3 = c3.poll().unwrap();
3301 let p1 = segment_data(&i1).unwrap().bytes.as_ptr();
3302 let p2 = segment_data(&i2).unwrap().bytes.as_ptr();
3303 let p3 = segment_data(&i3).unwrap().bytes.as_ptr();
3304
3305 assert_eq!(
3306 p1, p2,
3307 "cursor 2's segment payload must be the SAME allocation as cursor 1's"
3308 );
3309 assert_eq!(
3310 p2, p3,
3311 "cursor 3's segment payload must be the SAME allocation as cursor 1's"
3312 );
3313 assert_eq!(segment_data(&i1).unwrap().bytes.len(), 65536);
3314 }
3315
3316 // ===================== event log =======================================
3317
3318 use timed_metadata::{EventKind, SourcePayload};
3319
3320 /// A minimal `TimedEvent` for tests that don't care about the SCTE-35
3321 /// source payload itself — only about how the event *log* addresses and
3322 /// resolves it. `at`/`duration` are left `None`: this step's
3323 /// [`EventAnchor`] carries the resolution state, not `TimedEvent::at`.
3324 fn basic_event(id: u32) -> TimedEvent {
3325 TimedEvent {
3326 id: Some(id),
3327 kind: EventKind::BreakStart,
3328 at: None,
3329 duration: None,
3330 source: SourcePayload::Scte35 { raw: Vec::new() },
3331 }
3332 }
3333
3334 fn event_id(item: &EventCursorItem) -> Option<u32> {
3335 match item {
3336 EventCursorItem::Event(e) => e.event.id,
3337 _ => None,
3338 }
3339 }
3340
3341 /// Build real, valid (Parse/Serialize round-tripping) `splice_insert()`
3342 /// bytes carrying `pts_time`, via `scte35-splice`'s own builder +
3343 /// serializer — not hand-rolled/fabricated wire bytes. Used to drive
3344 /// `timed_metadata::Timeline::push_scte35`'s 33-bit wrap-unroll across a
3345 /// genuine wrap boundary (see `a_33_bit_pts_wrap_does_not_corrupt_event_log_ordering`).
3346 fn splice_insert_bytes(event_id: u32, pts_time: u64) -> Vec<u8> {
3347 use broadcast_common::Serialize;
3348 use scte35_splice::SpliceInfoSection;
3349 use scte35_splice::commands::AnyCommand;
3350 use scte35_splice::commands::splice_insert::SpliceInsert;
3351 use scte35_splice::time::SpliceTime;
3352
3353 let si = SpliceInsert {
3354 splice_event_id: event_id,
3355 out_of_network_indicator: true,
3356 splice_time: Some(SpliceTime::with_pts(pts_time)),
3357 ..SpliceInsert::default()
3358 };
3359 let section = SpliceInfoSection::new_clear(AnyCommand::SpliceInsert(si), &[]);
3360 section.to_bytes()
3361 }
3362
3363 // --- E1. events_between: half-open [from, to), boundaries exact -------
3364
3365 /// MUTATION VERIFIED: changing the upper-bound comparison in
3366 /// `Trunk::events_between`'s filter from `t.0 < to.0` to `t.0 <= to.0`
3367 /// (making the range closed instead of half-open) makes this test fail:
3368 /// `ids` becomes `[2, 3, 4]` (the boundary event at `to` is wrongly
3369 /// included) instead of the expected `[2, 3]`. Recompiled and re-run to
3370 /// confirm the failure, then reverted.
3371 #[test]
3372 fn events_between_returns_exactly_the_half_open_range() {
3373 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(8)));
3374 let writer = trunk.writer().unwrap();
3375
3376 for (id, ticks) in [(1u32, 1_000u64), (2, 2_000), (3, 3_000), (4, 4_000)] {
3377 writer.publish_event(basic_event(id), EventAnchor::Media(MediaTime(ticks)));
3378 }
3379
3380 let got = trunk.events_between(MediaTime(2_000), MediaTime(4_000));
3381 let ids: Vec<u32> = got.iter().map(|e| e.event.id.unwrap()).collect();
3382 assert_eq!(
3383 ids,
3384 vec![2, 3],
3385 "start (2_000) inclusive, end (4_000) exclusive"
3386 );
3387 }
3388
3389 // --- E2. a Segment-anchored entry resolves at PUBLISH time when the ---
3390 // --- boundary is already known ------------------------------------------
3391
3392 /// MUTATION VERIFIED: changing `EventLog::try_resolve`'s `Segment` arm
3393 /// to always return the anchor unresolved (`_ => anchor` in place of the
3394 /// `segment_starts` lookup) makes this test fail: `events_in_segment(3)`
3395 /// comes back empty instead of containing the published event, because
3396 /// the entry never leaves `EventAnchor::Segment`. Recompiled and re-run
3397 /// to confirm the failure, then reverted.
3398 #[test]
3399 fn segment_relative_event_resolves_at_publish_time_when_boundary_already_known() {
3400 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(8)));
3401 // Two separate writers, held at once: `note_segment_start` lives on
3402 // the segmenter's `SegmentWriter`, `publish_event` on the ingest
3403 // driver's `TrunkWriter` — exactly the split this step introduces.
3404 let writer = trunk.writer().unwrap();
3405 let segment_writer = trunk.segment_writer().unwrap();
3406
3407 segment_writer.note_segment_start(3, MediaTime(300_000));
3408 writer.publish_event(
3409 basic_event(9),
3410 EventAnchor::Segment {
3411 segment_number: 3,
3412 delta: 1_500,
3413 },
3414 );
3415
3416 let got = trunk.events_in_segment(3);
3417 assert_eq!(got.len(), 1);
3418 assert_eq!(got[0].event.id, Some(9));
3419 assert!(matches!(
3420 got[0].anchor,
3421 EventAnchor::Media(MediaTime(t)) if t == 301_500
3422 ));
3423 }
3424
3425 // --- E3. THE B1 SEGMENT CASE: a segment-relative event resolves to ----
3426 // --- the segment it actually named, not whichever segment is open -----
3427
3428 /// MUTATION VERIFIED: removing the `if n == segment_number` guard in
3429 /// `EventLog::note_segment_start` (resolving *every* pending `Segment`
3430 /// entry against whichever boundary arrives, regardless of which
3431 /// segment it targets) makes this test fail at the first assertion:
3432 /// after `note_segment_start(1, MediaTime(0))` — segment 1, NOT the
3433 /// event's actual target segment 2 — the entry is wrongly resolved to
3434 /// `MediaTime(1_000)` (segment 1's start + delta) instead of staying
3435 /// `EventAnchor::Segment { segment_number: 2, .. }`, so the
3436 /// `matches!(entry.anchor, EventAnchor::Segment { segment_number: 2, .. })`
3437 /// assertion fails. Recompiled and re-run to confirm the failure, then
3438 /// reverted.
3439 #[test]
3440 fn segment_relative_event_resolves_to_the_named_segment_not_whichever_is_open() {
3441 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(8)));
3442 let writer = trunk.writer().unwrap();
3443 let segment_writer = trunk.segment_writer().unwrap();
3444 let mut cursor = trunk.subscribe_events();
3445
3446 // The event targets segment 2 specifically, delta 1_000 after ITS
3447 // start — published before ANY segment boundary is known.
3448 writer.publish_event(
3449 basic_event(42),
3450 EventAnchor::Segment {
3451 segment_number: 2,
3452 delta: 1_000,
3453 },
3454 );
3455
3456 // Segment 1 — a DIFFERENT, "currently open" segment — reports its
3457 // start first. This must NOT resolve the segment-2-targeted event.
3458 segment_writer.note_segment_start(1, MediaTime(0));
3459
3460 let item = cursor.poll().unwrap();
3461 let entry = match item {
3462 EventCursorItem::Event(e) => e,
3463 other => panic!("expected Event, got {other:?}"),
3464 };
3465 assert!(
3466 matches!(
3467 entry.anchor,
3468 EventAnchor::Segment {
3469 segment_number: 2,
3470 delta: 1_000
3471 }
3472 ),
3473 "must stay pending on segment 2 — segment 1 being open must not \
3474 resolve it against the wrong boundary: {:?}",
3475 entry.anchor
3476 );
3477 assert!(
3478 trunk.events_in_segment(2).is_empty(),
3479 "not resolved yet: must not appear under segment 2 either"
3480 );
3481 assert!(trunk.events_in_segment(1).is_empty());
3482
3483 // Now segment 2's own start arrives: resolves in place, to the
3484 // RIGHT segment's start + delta.
3485 segment_writer.note_segment_start(2, MediaTime(90_000));
3486
3487 let in_seg2 = trunk.events_in_segment(2);
3488 assert_eq!(in_seg2.len(), 1);
3489 assert_eq!(in_seg2[0].event.id, Some(42));
3490 assert!(matches!(
3491 in_seg2[0].anchor,
3492 EventAnchor::Media(MediaTime(t)) if t == 91_000
3493 ));
3494 assert!(
3495 trunk.events_in_segment(1).is_empty(),
3496 "must not ALSO appear under segment 1"
3497 );
3498 }
3499
3500 // --- E4. THE B1 CRUX: a UTC-only event stays honestly unanchored ------
3501 // --- until a TimeAnchor arrives, then resolves correctly ---------------
3502
3503 /// MUTATION VERIFIED: changing `EventLog::try_resolve`'s `Utc` arm to
3504 /// fabricate `EventAnchor::Media(MediaTime(0))` whenever no
3505 /// `time_anchor` is set yet (in place of returning the anchor
3506 /// unresolved) — i.e. reintroducing the exact B1 bug this design
3507 /// exists to prevent — makes this test fail at the first assertion:
3508 /// `entry.anchor` is `EventAnchor::Media(MediaTime(0))` instead of the
3509 /// expected `EventAnchor::Utc { utc_epoch_ms: 5_000 }`, so the
3510 /// `matches!` assertion fails. Recompiled and re-run to confirm the
3511 /// failure, then reverted.
3512 #[test]
3513 fn utc_only_event_stays_unanchored_until_a_time_anchor_arrives() {
3514 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(8)));
3515 let writer = trunk.writer().unwrap();
3516 let segment_writer = trunk.segment_writer().unwrap();
3517 let mut cursor = trunk.subscribe_events();
3518
3519 // A GPS/UTC-scheduled event (SCTE-35 splice_schedule.utc_splice_time
3520 // semantics, §9.7.4) with no media anchor yet.
3521 writer.publish_event(
3522 basic_event(7),
3523 EventAnchor::Utc {
3524 utc_epoch_ms: 5_000,
3525 },
3526 );
3527
3528 let item = cursor.poll().unwrap();
3529 let entry = match item {
3530 EventCursorItem::Event(e) => e,
3531 other => panic!("expected Event, got {other:?}"),
3532 };
3533 assert!(
3534 matches!(
3535 entry.anchor,
3536 EventAnchor::Utc {
3537 utc_epoch_ms: 5_000
3538 }
3539 ),
3540 "must stay honestly unanchored — NO fabricated media time: {:?}",
3541 entry.anchor
3542 );
3543 // Nothing to filter a media time against yet: the point-in-time
3544 // query must not surface it either.
3545 assert!(
3546 trunk
3547 .events_between(MediaTime(0), MediaTime(u64::MAX))
3548 .is_empty(),
3549 "an unanchored event must not appear in a media-time query"
3550 );
3551
3552 // An anchor arrives: pts 0 == epoch 1_000ms (`TimeAnchor`'s own
3553 // convention), so epoch 5_000ms is 4_000ms == 360_000 ticks later.
3554 segment_writer.set_time_anchor(TimeAnchor {
3555 pts_90k: 0,
3556 utc_epoch_ms: 1_000,
3557 });
3558
3559 let resolved = trunk.events_between(MediaTime(0), MediaTime(u64::MAX));
3560 assert_eq!(resolved.len(), 1);
3561 assert_eq!(resolved[0].event.id, Some(7));
3562 assert!(
3563 matches!(resolved[0].anchor, EventAnchor::Media(MediaTime(t)) if t == 360_000),
3564 "expected MediaTime(360_000), got {:?}",
3565 resolved[0].anchor
3566 );
3567 }
3568
3569 // --- E5. a 33-bit PTS wrap does not corrupt event log ordering --------
3570 // --- (reuses timed_metadata::Timeline's unroll; does not hand-roll it) -
3571
3572 /// MUTATION VERIFIED: re-introducing a 33-bit mask on an
3573 /// already-unrolled `MediaTime` in `EventLog::try_resolve`'s `Media`
3574 /// arm (`EventAnchor::Media(MediaTime(t)) => EventAnchor::Media(MediaTime(t
3575 /// & ((1u64 << 33) - 1)))` in place of the pass-through `anchor`) makes
3576 /// this test fail: `ev2`'s post-wrap absolute tick value
3577 /// (`(1u64 << 33) + 5`) exceeds 33 bits, so it is stored truncated to
3578 /// `5` instead of the value `Timeline` actually computed, and
3579 /// `matches!(got[1].anchor, EventAnchor::Media(t) if t.0 == at2.0)`
3580 /// fails (stored `5` != `at2.0` ≈ `2^33 + 5`). (The earlier
3581 /// `at2.0 > at1.0` assertion, which only reads `Timeline`'s local return
3582 /// value, does NOT catch this mutation — a stored-value mutation only
3583 /// shows up in what the log hands back, which is exactly why this test
3584 /// asserts against `got[..].anchor`, not just `at1`/`at2`.) Recompiled
3585 /// and re-run to confirm the failure, then reverted.
3586 #[test]
3587 fn a_33_bit_pts_wrap_does_not_corrupt_event_log_ordering() {
3588 const PTS_WRAP: u64 = 1u64 << 33;
3589
3590 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(8)));
3591 let writer = trunk.writer().unwrap();
3592 let mut timeline = timed_metadata::Timeline::new();
3593
3594 // Event 1: a PTS 10 ticks before the 33-bit wrap point.
3595 let before_wrap = splice_insert_bytes(1, PTS_WRAP - 10);
3596 let ev1 = timeline.push_scte35(&before_wrap).unwrap();
3597 let at1 = ev1.at.unwrap();
3598 writer.publish_event(ev1, EventAnchor::Media(at1));
3599
3600 // Event 2: a small RAW PTS after the wrap. `Timeline` must unroll
3601 // this into a value larger than `at1`, not a small one.
3602 let after_wrap = splice_insert_bytes(2, 5);
3603 let ev2 = timeline.push_scte35(&after_wrap).unwrap();
3604 let at2 = ev2.at.unwrap();
3605 writer.publish_event(ev2, EventAnchor::Media(at2));
3606
3607 assert!(
3608 at2.0 > at1.0,
3609 "Timeline itself must unroll monotonically: at1={}, at2={}",
3610 at1.0,
3611 at2.0
3612 );
3613
3614 // The event log must store EXACTLY the MediaTime `Timeline` already
3615 // unrolled — no re-derivation, re-masking, or truncation of an
3616 // already-unrolled value anywhere in this module's storage/
3617 // resolution path. (Publish-order preservation across a wrap is
3618 // trivial regardless of the anchor's value — `VecDeque` iteration
3619 // order does not depend on it — so the real assertion here is
3620 // value-exactness, not position.)
3621 let got = trunk.events_between(MediaTime(0), MediaTime(u64::MAX));
3622 assert_eq!(got.len(), 2);
3623 assert_eq!(got[0].event.id, Some(1));
3624 assert_eq!(got[1].event.id, Some(2));
3625 assert!(
3626 matches!(got[0].anchor, EventAnchor::Media(t) if t.0 == at1.0),
3627 "event 1's stored anchor must equal Timeline's unrolled value \
3628 exactly, got {:?}",
3629 got[0].anchor
3630 );
3631 assert!(
3632 matches!(got[1].anchor, EventAnchor::Media(t) if t.0 == at2.0),
3633 "event 2's stored (post-wrap) anchor must equal Timeline's \
3634 unrolled value exactly — not re-masked back into 33 bits, got {:?}",
3635 got[1].anchor
3636 );
3637 }
3638
3639 // --- E6. the event log is bounded: flooding cannot grow memory --------
3640 // --- without limit -------------------------------------------------------
3641
3642 /// MUTATION VERIFIED: removing the eviction check in `EventLog::push`
3643 /// (replacing `if self.entries.len() == self.capacity { .. }` with a
3644 /// no-op) makes `trunk.event_len()` grow well past the configured cap
3645 /// (`3`) instead of staying bounded — the in-loop assertion fails on
3646 /// the first over-capacity iteration. Recompiled and re-run to confirm
3647 /// the failure, then reverted.
3648 #[test]
3649 fn event_log_is_bounded_under_flood() {
3650 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(3), nz(8)));
3651 let writer = trunk.writer().unwrap();
3652
3653 for i in 0u32..50_000 {
3654 writer.publish_event(basic_event(i), EventAnchor::Media(MediaTime(u64::from(i))));
3655 assert!(
3656 trunk.event_len() <= 3,
3657 "event log exceeded its cap mid-flood"
3658 );
3659 }
3660 assert_eq!(trunk.event_len(), 3);
3661 }
3662
3663 // --- E7. event cursor lag is reported in-band with an accurate --------
3664 // --- skipped count; the writer never blocks -----------------------------
3665
3666 /// MUTATION VERIFIED: changing the `skipped` computation in
3667 /// `EventCursor::poll`'s lag branch from `log.base - self.consumed` to
3668 /// `log.base - self.consumed + 1` makes this test fail: expected
3669 /// `skipped: 6`, got `skipped: 7`. Recompiled and re-run to confirm the
3670 /// failure, then reverted.
3671 #[test]
3672 fn event_cursor_lag_is_reported_with_an_accurate_skipped_count_writer_never_blocks() {
3673 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(3), nz(8)));
3674 let mut cursor = trunk.subscribe_events();
3675 let writer = trunk.writer().unwrap();
3676
3677 // Capacity 3, publish 9: 6 evicted before the cursor ever reads.
3678 // Never blocks — there is no wait-for-reader path in
3679 // `EventLog::push`.
3680 for i in 0u32..9 {
3681 writer.publish_event(basic_event(i), EventAnchor::Media(MediaTime(u64::from(i))));
3682 }
3683 assert_eq!(
3684 trunk.event_len(),
3685 3,
3686 "writer unblocked: event log stayed bounded"
3687 );
3688
3689 let first = cursor.poll().unwrap();
3690 assert!(
3691 matches!(first, EventCursorItem::Lagged { skipped: 6 }),
3692 "expected Lagged{{skipped: 6}}, got {first:?}"
3693 );
3694
3695 // The remaining 3 (ids 6, 7, 8) must still be readable, in order.
3696 let mut ids = Vec::new();
3697 for _ in 0..3 {
3698 ids.push(event_id(&cursor.poll().unwrap()).unwrap());
3699 }
3700 assert_eq!(ids, vec![6, 7, 8]);
3701 assert!(cursor.poll().is_none());
3702 }
3703
3704 // --- E8. `epoch_ms_to_media` really is the inverse of ------------------
3705 // --- `TimeAnchor::media_to_epoch_ms` -----------------------------------
3706
3707 /// A wall-clock anchor with room on *both* sides of `pts_90k`, so a sign
3708 /// error in the (signed) delta shows up rather than being clipped away:
3709 /// 10 s of media already elapsed, mapped to a realistic epoch instant.
3710 fn round_trip_anchor() -> TimeAnchor {
3711 TimeAnchor {
3712 pts_90k: 900_000, // 10 s at 90 kHz
3713 utc_epoch_ms: 1_700_000_000_000, // ~2023-11-14T22:13:20Z
3714 }
3715 }
3716
3717 /// Ticks of the 90 kHz media clock per millisecond of the epoch clock.
3718 ///
3719 /// Derived, not asserted: the media clock is [`PTS_HZ`] ticks/second and
3720 /// `utc_epoch_ms` counts milliseconds, i.e. thousandths of a second, so
3721 /// one millisecond spans `PTS_HZ / 1000` ticks.
3722 const TICKS_PER_EPOCH_MS: u64 = PTS_HZ / 1000;
3723
3724 /// The exact worst-case `media -> epoch_ms -> media` error, in ticks.
3725 ///
3726 /// **Derivation (not a tuned constant).**
3727 /// [`TimeAnchor::media_to_epoch_ms`] computes
3728 /// `delta_ticks * 1000 / PTS_HZ`, i.e. `delta_ticks / TICKS_PER_EPOCH_MS`,
3729 /// in integer arithmetic — Rust integer division truncates toward zero,
3730 /// so it discards a remainder `r` with `|r| <= TICKS_PER_EPOCH_MS - 1`.
3731 /// `epoch_ms_to_media` then multiplies the surviving whole milliseconds
3732 /// back up by `TICKS_PER_EPOCH_MS`, reconstructing `delta_ticks - r`
3733 /// exactly. The round-trip error is therefore *precisely* that discarded
3734 /// remainder: at most `TICKS_PER_EPOCH_MS - 1` == 89 ticks, i.e. strictly
3735 /// less than one millisecond. `media_round_trip_is_lossy_by_at_most_one_
3736 /// millisecond` additionally asserts this bound is **tight** (some input
3737 /// attains exactly 89), so it cannot silently be loosened into a
3738 /// tolerance that hides a real error.
3739 const MEDIA_ROUND_TRIP_MAX_TICKS: u64 = TICKS_PER_EPOCH_MS - 1;
3740
3741 /// The `epoch_ms -> media -> epoch_ms` direction is **exact** — the media
3742 /// clock is finer-grained than the millisecond clock (90 ticks per ms),
3743 /// so no information is lost going to ticks and back. Asserted with
3744 /// equality, no tolerance.
3745 ///
3746 /// MUTATION VERIFIED: flipping the sign of the delta in
3747 /// `epoch_ms_to_media` (`i128::from(anchor.utc_epoch_ms) -
3748 /// i128::from(utc_epoch_ms)` in place of the correct
3749 /// `i128::from(utc_epoch_ms) - i128::from(anchor.utc_epoch_ms)`) makes
3750 /// this test fail on the first non-zero offset: for `+1` ms the
3751 /// round-tripped epoch comes back as `1699999999999` instead of
3752 /// `1700000000001`. **Second mutation, also verified:** changing the
3753 /// scale conversion from `* PTS_HZ / 1000` to `* PTS_HZ * 1000` fails the
3754 /// same assertion with `1700001000000` instead of `1700000000001`. So
3755 /// both the *sign* and the *magnitude* of the inverse are pinned, not
3756 /// just its shape. Recompiled and re-run to confirm each failure, then
3757 /// reverted.
3758 #[test]
3759 fn epoch_ms_round_trip_through_media_time_is_exact() {
3760 let anchor = round_trip_anchor();
3761
3762 // Offsets in ms from the anchor's own epoch instant. Zero, both
3763 // signs at ±1 ms and ±1 s, a full day forward, a backward offset
3764 // that lands well clear of the clamp, and one large enough that
3765 // `delta_ms * PTS_HZ` (2e14 * 9e4 = 1.8e19) exceeds `i64::MAX`
3766 // (~9.2e18) — the case that exercises the `i128` widening.
3767 for offset_ms in [
3768 0i64,
3769 1,
3770 -1,
3771 1_000,
3772 -1_000,
3773 86_400_000,
3774 -9_000,
3775 200_000_000_000_000,
3776 ] {
3777 let epoch_ms = anchor.utc_epoch_ms + offset_ms;
3778 let media = epoch_ms_to_media(&anchor, epoch_ms);
3779 let back = anchor.media_to_epoch_ms(media);
3780 assert_eq!(
3781 back, epoch_ms,
3782 "epoch_ms -> media -> epoch_ms must be EXACT at offset {offset_ms} ms \
3783 (media = {media:?})"
3784 );
3785 }
3786 }
3787
3788 /// The `media -> epoch_ms -> media` direction is **lossy**, by a bounded
3789 /// and derived amount: the media clock is 90× finer than the millisecond
3790 /// clock, so sub-millisecond tick precision cannot survive the trip. See
3791 /// [`MEDIA_ROUND_TRIP_MAX_TICKS`] for the derivation. This test also
3792 /// pins the bound as *tight*, so it is a real property and not a loose
3793 /// tolerance hiding an error.
3794 ///
3795 /// MUTATION VERIFIED: flipping the sign of the delta in
3796 /// `epoch_ms_to_media` (as in
3797 /// `epoch_ms_round_trip_through_media_time_is_exact`'s note) makes this
3798 /// test fail at the first offset that is a whole number of milliseconds
3799 /// away from the anchor: at media offset `+90` ticks the value comes
3800 /// back as `899_910` instead of `900_090`, a diff of `180` ticks, so the
3801 /// `diff <= MEDIA_ROUND_TRIP_MAX_TICKS` (89) assertion fails.
3802 /// **Second mutation, also verified:** the `* PTS_HZ * 1000` scale error
3803 /// fails the same assertion with a diff of `89_999_910` ticks. Recompiled
3804 /// and re-run to confirm each failure, then reverted.
3805 #[test]
3806 fn media_round_trip_is_lossy_by_at_most_one_millisecond() {
3807 let anchor = round_trip_anchor();
3808 let mut worst = 0u64;
3809
3810 // Tick offsets from the anchor's own `pts_90k`. Both signs, values
3811 // that are and are not whole multiples of TICKS_PER_EPOCH_MS (so the
3812 // truncated remainder is genuinely exercised), the exact worst-case
3813 // remainder on each side (±89), and a large offset well past the
3814 // i64/i128 boundary region.
3815 for offset_ticks in [
3816 0i64,
3817 1,
3818 -1,
3819 89,
3820 -89,
3821 90,
3822 -90,
3823 91,
3824 -91,
3825 18_000_000_000_000_037,
3826 ] {
3827 let media = MediaTime((anchor.pts_90k as i64 + offset_ticks) as u64);
3828 let epoch_ms = anchor.media_to_epoch_ms(media);
3829 let back = epoch_ms_to_media(&anchor, epoch_ms);
3830 let diff = media.0.abs_diff(back.0);
3831 assert!(
3832 diff <= MEDIA_ROUND_TRIP_MAX_TICKS,
3833 "media -> epoch_ms -> media lost {diff} ticks at offset \
3834 {offset_ticks} (bound is {MEDIA_ROUND_TRIP_MAX_TICKS}, i.e. \
3835 < 1 ms): {media:?} -> {epoch_ms} -> {back:?}"
3836 );
3837 worst = worst.max(diff);
3838 }
3839
3840 // The bound is TIGHT: the ±89-tick cases attain it exactly. Without
3841 // this, `MEDIA_ROUND_TRIP_MAX_TICKS` could be quietly raised to
3842 // paper over a genuine arithmetic error and the test above would
3843 // still pass.
3844 assert_eq!(
3845 worst, MEDIA_ROUND_TRIP_MAX_TICKS,
3846 "the derived bound must be attained, not merely respected — \
3847 otherwise it is a loose tolerance, not a property"
3848 );
3849 }
3850
3851 /// `epoch_ms_to_media`'s `clamp(0, u64::MAX)` for an epoch instant far
3852 /// enough *before* the anchor that the implied media time would be
3853 /// negative.
3854 ///
3855 /// **This documents clamping as SAFE, not CORRECT** — they are different
3856 /// claims and this test asserts the weaker, true one. A negative media
3857 /// time is simply not representable in `MediaTime(u64)`, so no return
3858 /// value here can be right: clamping to `0` reports "at the very start
3859 /// of this trunk's timeline", which is *not* the instant asked for, and
3860 /// the round trip provably does not recover the input (asserted below).
3861 /// What the clamp does buy is that the failure is bounded and obvious
3862 /// rather than catastrophic: an unchecked `as u64` cast of a negative
3863 /// value would wrap to something near `u64::MAX` — an event appearing
3864 /// scheduled ~6.5 million years in the future, which is exactly the
3865 /// silent wrong-instant class B1 is about. Clamping keeps a
3866 /// pre-origin event in the past (where a scheduler treats it as already
3867 /// elapsed) instead of the unreachable future.
3868 ///
3869 /// If pre-origin scheduled events turn out to be real rather than
3870 /// pathological, the *honest* fix is not a different clamp value — it is
3871 /// to leave the entry `EventAnchor::Utc` (unresolved), exactly as an
3872 /// event with no anchor at all stays unresolved. That would be an
3873 /// additive change to `try_resolve`/`set_time_anchor`, not a change to
3874 /// this helper's contract.
3875 #[test]
3876 fn epoch_before_the_timeline_origin_clamps_to_zero_which_is_safe_not_correct() {
3877 let anchor = round_trip_anchor();
3878
3879 // 20 s before the anchor's epoch, but only 10 s of media has
3880 // elapsed at the anchor — so the implied media time is -10 s.
3881 let epoch_ms = anchor.utc_epoch_ms - 20_000;
3882 let media = epoch_ms_to_media(&anchor, epoch_ms);
3883
3884 assert_eq!(
3885 media,
3886 MediaTime(0),
3887 "a pre-origin epoch must clamp to the start of the timeline"
3888 );
3889
3890 // Bounded-and-obvious, not catastrophic: emphatically NOT a wrapped
3891 // near-`u64::MAX` value masquerading as the far future.
3892 assert!(
3893 media.0 < u64::from(u32::MAX),
3894 "must not have wrapped into the far future: {media:?}"
3895 );
3896
3897 // And it is genuinely NOT correct: the round trip does not recover
3898 // the input, because the requested instant is unrepresentable.
3899 let back = anchor.media_to_epoch_ms(media);
3900 assert_ne!(
3901 back, epoch_ms,
3902 "clamping is lossy by construction — this asserts the honest \
3903 claim (safe) rather than the false one (correct)"
3904 );
3905 assert_eq!(
3906 back,
3907 anchor.utc_epoch_ms - 10_000,
3908 "clamped media time 0 maps back to the timeline origin (10 s \
3909 before the anchor), not to the requested instant"
3910 );
3911 }
3912
3913 // === Step 3b-iv: live-part log + reader-wake primitive =================
3914
3915 fn part_entry(byte: u8, segment_number: u32, part_index: u32) -> PartEntry {
3916 PartEntry::new(
3917 Bytes::from(vec![byte; 8]),
3918 segment_number,
3919 part_index,
3920 Duration::from_millis(200),
3921 part_index == 0,
3922 )
3923 }
3924
3925 /// The LL-HLS property this whole step exists for: a part must be
3926 /// addressable and readable **before** its parent segment closes — if
3927 /// this does not hold, nothing else in this step matters (blocking
3928 /// reload has nothing to answer with).
3929 ///
3930 /// MUTATION VERIFIED: commenting out `state.parts.push(entry);` in
3931 /// `SegmentWriter::publish_part` (simulating "the part never actually
3932 /// lands in the log") makes `trunk.part_bytes(9, 0)` return `None`
3933 /// instead of `Some(..)`, failing the `expect` below. Recompiled and
3934 /// re-run to confirm the failure, then reverted.
3935 #[test]
3936 fn part_is_addressable_and_readable_before_its_parent_segment_closes() {
3937 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(8)));
3938 let writer = trunk.segment_writer().unwrap();
3939
3940 // Segment 9 has never been closed — no `publish_segment` call for it
3941 // anywhere in this test.
3942 assert!(
3943 trunk.last_closed_segment().is_none(),
3944 "sanity check: nothing has closed yet"
3945 );
3946
3947 writer.publish_part(part_entry(0xAB, 9, 0));
3948
3949 let bytes = trunk
3950 .part_bytes(9, 0)
3951 .expect("a published part of an open segment must be addressable now");
3952 assert_eq!(bytes, Bytes::from(vec![0xAB; 8]));
3953
3954 // Still true: segment 9 has still never closed.
3955 assert!(
3956 trunk.last_closed_segment().is_none(),
3957 "the part landed without any segment ever closing"
3958 );
3959 }
3960
3961 /// A waiter blocked on a not-yet-existing part wakes once it is
3962 /// published, and resolves to exactly that part — not merely "wakes",
3963 /// which a mutation could satisfy vacuously if `part_bytes` mismatched
3964 /// the wrong entry (this test publishes a decoy part first to make
3965 /// "the right one" a real assertion).
3966 ///
3967 /// MUTATION VERIFIED (two independent mutations, each reverted after
3968 /// confirming failure):
3969 /// 1. Removing `self.trunk.progress.notify(usize::MAX);` from
3970 /// `SegmentWriter::publish_part` makes `listener.wait_deadline(..)`
3971 /// time out (`false`) instead of waking (`true`) within the 2 s bound
3972 /// used below — the first assertion fails.
3973 /// 2. Changing `Trunk::part_bytes`'s filter from
3974 /// `p.segment_number == segment_number && p.part_index == part_index`
3975 /// to drop the `part_index` half (matching on `segment_number` alone)
3976 /// makes the final `assert_eq!` fail: with the decoy part (index 1)
3977 /// published first, `part_bytes(9, 0)` would resolve to the decoy's
3978 /// `0xCC` bytes instead of the awaited part's `0xAB` bytes.
3979 #[test]
3980 fn waiter_is_woken_when_the_awaited_part_lands_and_resolves_to_it() {
3981 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(8)));
3982 let writer = Arc::new(trunk.segment_writer().unwrap());
3983
3984 // Register BEFORE re-checking/waiting — the documented no-missed-
3985 // wakeup ordering.
3986 let listener = trunk.listen().expect("first registration must succeed");
3987 assert!(trunk.part_bytes(9, 0).is_none(), "not published yet");
3988
3989 let bg_writer = Arc::clone(&writer);
3990 let handle = thread::spawn(move || {
3991 thread::sleep(Duration::from_millis(50));
3992 // A decoy: a different part of the same segment, published
3993 // first. If `part_bytes` ever matched on `segment_number` alone,
3994 // this would be the value wrongly returned for a request for
3995 // part 0.
3996 bg_writer.publish_part(part_entry(0xCC, 9, 1));
3997 bg_writer.publish_part(part_entry(0xAB, 9, 0));
3998 });
3999
4000 // Deliberately generous. The claim under test is "the listener wakes
4001 // rather than parking forever", NOT "it wakes inside N seconds": the
4002 // publish happens on another thread, so any tight upper bound is a
4003 // bound on the *machine's* scheduling, not on this code. A 2s bound
4004 // here failed once during a loaded full-workspace run and passed on
4005 // 38 consecutive idle runs -- a false red that trains people to
4006 // re-run the suite. 60s still fails instantly if the wake channel
4007 // genuinely never fires, which is the only failure worth reporting.
4008 let woken = listener.wait_deadline(std::time::Instant::now() + Duration::from_secs(60));
4009 assert!(
4010 woken,
4011 "listener must wake on publish_part, not park forever"
4012 );
4013 handle.join().unwrap();
4014
4015 let bytes = trunk
4016 .part_bytes(9, 0)
4017 .expect("the awaited part must now be readable");
4018 assert_eq!(
4019 bytes,
4020 Bytes::from(vec![0xAB; 8]),
4021 "must resolve to the awaited part (index 0), not the decoy (index 1)"
4022 );
4023 }
4024
4025 /// A waiter whose target never arrives is bounded by its own deadline —
4026 /// it does not park forever. This composes with
4027 /// `crate::egress::AwaitPolicy`'s deadline exactly the same way: the
4028 /// caller converts its own bound to a `std::time::Instant` and passes it
4029 /// here.
4030 ///
4031 /// MUTATION VERIFIED: changing `ProgressListener::wait_deadline`'s body
4032 /// from `listener.wait_deadline(deadline).is_some()` to unconditionally
4033 /// `true` makes this test's `assert!(!woken, ..)` fail — `woken` is
4034 /// `true` even though nothing was ever published. Recompiled and re-run
4035 /// to confirm the failure, then reverted.
4036 #[test]
4037 fn waiter_whose_target_never_arrives_is_bounded_not_parked_forever() {
4038 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(8)));
4039 let listener = trunk.listen().unwrap();
4040
4041 let start = std::time::Instant::now();
4042 let woken = listener.wait_deadline(start + Duration::from_millis(150));
4043 let elapsed = start.elapsed();
4044
4045 assert!(!woken, "must report timeout, not a fabricated wake-up");
4046 // Same reasoning as the generous bound in
4047 // `awaited_part_wakes_its_listener`: this asserts "returns rather
4048 // than parking forever", and any tight bound measures the machine.
4049 // The real assertion is `!woken` above; this one only catches a hang.
4050 assert!(
4051 elapsed < Duration::from_secs(60),
4052 "must actually return at the deadline, not hang: took {elapsed:?}"
4053 );
4054 }
4055
4056 /// The hard invariant this whole file exists to preserve, extended to
4057 /// the wake channel: `publish_part`/`publish_segment` must still
4058 /// complete even with a waiter registered and never serviced (no one
4059 /// ever calls `wait`/`.await`/drops it) — a slow or vanished reader must
4060 /// never stall the writer. Proven the same way this file's existing
4061 /// `StallIngest`-blocks proof works, but for the opposite claim: a
4062 /// background-thread `publish_*` call, and a bounded `recv_timeout`
4063 /// proving it completed promptly.
4064 ///
4065 /// This specific guarantee is structural, not something a local
4066 /// mutation of this crate's code can plausibly violate: `publish_part`/
4067 /// `publish_segment` call `Event::notify(usize::MAX)`, which by
4068 /// `event_listener`'s own documented contract wakes registered listeners
4069 /// without waiting for any of them to resume — there is no "wait for
4070 /// the listener to be serviced" code path in this module to remove.
4071 /// Reaching a blocking wake-up would require swapping the whole
4072 /// primitive for a different one (the architectural choice already
4073 /// argued in this module's docs), not a one-line mutation, so no
4074 /// mutation transcript is claimed for this test — see this crate's
4075 /// convention that a structural property is reported as such rather
4076 /// than backed by an invented mutation.
4077 #[test]
4078 fn writer_never_blocks_with_a_registered_never_serviced_waiter() {
4079 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(8)));
4080 let writer = Arc::new(trunk.segment_writer().unwrap());
4081
4082 // Registered, kept alive for the whole test, and never waited on or
4083 // dropped before the assertions below run.
4084 let _never_serviced = trunk.listen().unwrap();
4085
4086 let (done_tx, done_rx) = mpsc::channel();
4087 let bg_writer = Arc::clone(&writer);
4088 thread::spawn(move || {
4089 bg_writer.publish_part(part_entry(1, 1, 0));
4090 bg_writer.publish_segment(segment_entry(2, 1));
4091 done_tx.send(()).unwrap();
4092 });
4093
4094 // HANG GUARD (issue #807): the property under test is "never blocks",
4095 // i.e. these calls return almost immediately (`Event::notify`
4096 // doesn't wait for listeners to resume) -- a stuck/blocked writer
4097 // here is the only thing this should ever catch, so raised for
4098 // load-tolerance rather than left as a timing claim.
4099 done_rx.recv_timeout(Duration::from_secs(60)).expect(
4100 "publish_part/publish_segment must complete promptly even with \
4101 a live, never-serviced waiter registered",
4102 );
4103 }
4104
4105 /// The waiter set itself is bounded — a flood of `listen()` calls cannot
4106 /// grow memory without limit, and reuses `part_capacity` rather than a
4107 /// sixth, independent knob.
4108 ///
4109 /// MUTATION VERIFIED: replacing `Trunk::listen`'s cap check (the
4110 /// `if current >= self.part_waiter_cap { return None; }` loop) with a
4111 /// version that always registers (never returns `None`) makes the
4112 /// `assert!(trunk.listen().is_none(), ..)` below fail — the call
4113 /// succeeds instead of being refused at the cap. Recompiled and re-run
4114 /// to confirm the failure, then reverted.
4115 #[test]
4116 fn waiter_set_is_bounded_a_flood_of_listen_calls_cannot_grow_without_limit() {
4117 let cap = nz(4).get();
4118 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(cap)));
4119
4120 // Fill exactly to the cap, keeping every registration alive.
4121 let mut held: Vec<ProgressListener> = Vec::new();
4122 for _ in 0..cap {
4123 held.push(trunk.listen().expect("must succeed up to the cap"));
4124 }
4125 assert_eq!(trunk.waiter_count(), cap);
4126
4127 // One more must be refused, not silently over-admitted.
4128 assert!(
4129 trunk.listen().is_none(),
4130 "must refuse a registration beyond part_capacity"
4131 );
4132
4133 // A flood of register-then-immediately-drop calls (no one keeping
4134 // them alive) must never push the live count past the cap, however
4135 // many times it runs.
4136 for _ in 0..50_000 {
4137 let l = trunk.listen();
4138 assert!(
4139 trunk.waiter_count() <= cap,
4140 "waiter count exceeded part_capacity mid-flood"
4141 );
4142 drop(l);
4143 }
4144 assert_eq!(
4145 trunk.waiter_count(),
4146 cap,
4147 "the held registrations are still exactly at the cap"
4148 );
4149
4150 // Releasing one held slot frees exactly one registration.
4151 held.pop();
4152 assert_eq!(trunk.waiter_count(), cap - 1);
4153 assert!(
4154 trunk.listen().is_some(),
4155 "a released slot must be re-usable"
4156 );
4157 }
4158
4159 /// The decided close-behaviour, asserted: a part remains addressable via
4160 /// `part_bytes` after its parent segment closes (this trunk's `Trunk`
4161 /// does not evict/transform parts on `publish_segment`), right up until
4162 /// `part_capacity`'s ordinary eviction reclaims it — at which point a
4163 /// client requesting that same part gets `None`, indistinguishable from
4164 /// "never existed", exactly like every other ring's eviction in this
4165 /// module.
4166 ///
4167 /// MUTATION VERIFIED: adding an eviction step to `publish_segment` that
4168 /// removes every `PartLog` entry whose `segment_number` matches the
4169 /// just-closed segment (simulating the rejected "evict a segment's
4170 /// parts the instant it closes" alternative documented in this module's
4171 /// docs) makes the first `part_bytes(1, 0)` assertion below fail
4172 /// immediately after `publish_segment` — it returns `None` instead of
4173 /// `Some(..)`. Recompiled and re-run to confirm the failure, then
4174 /// reverted.
4175 #[test]
4176 fn parts_remain_addressable_after_segment_close_until_ordinary_eviction_reclaims_them() {
4177 let part_cap = nz(4).get();
4178 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(part_cap)));
4179 let writer = trunk.segment_writer().unwrap();
4180
4181 writer.publish_part(part_entry(0xAB, 1, 0));
4182 writer.publish_segment(segment_entry(1, 1));
4183
4184 // The part a client just watched roll into a closed segment is
4185 // still `Some` — the same answer as before the close.
4186 assert_eq!(
4187 trunk.part_bytes(1, 0),
4188 Some(Bytes::from(vec![0xAB; 8])),
4189 "a just-closed segment's part must still be individually fetchable"
4190 );
4191 assert_eq!(trunk.last_closed_segment(), Some(1));
4192
4193 // Flood the part ring with `part_cap` more entries for an unrelated
4194 // segment — enough to evict the original part via ordinary
4195 // capacity-based eviction, with no further segment closes involved.
4196 for i in 0..part_cap as u32 {
4197 writer.publish_part(part_entry(0xFF, 99, i));
4198 }
4199
4200 assert!(
4201 trunk.part_bytes(1, 0).is_none(),
4202 "the part is gone once ordinary part_capacity eviction reclaims \
4203 it — NOT because its segment closed, but because the ring's own \
4204 bound was exceeded, exactly like every other ring in this module"
4205 );
4206 }
4207
4208 // === Issue #781: track-set snapshot + generation counter ==============
4209
4210 /// A freshly-minted `Trunk` announces no tracks yet — `set_tracks` has
4211 /// never been called, so there is nothing to seed `tracks()`/
4212 /// `track_generation()` from other than the empty defaults `Trunk::new`
4213 /// establishes.
4214 ///
4215 /// MUTATION VERIFIED: changing `Trunk::new`'s `track_generation: 0` to
4216 /// `track_generation: 1` makes the second `assert_eq!` below fail — it
4217 /// reads back `1` instead of `0`. Recompiled and re-run to confirm the
4218 /// failure, then reverted.
4219 #[test]
4220 fn fresh_trunk_has_no_tracks_and_generation_zero() {
4221 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(8)));
4222 assert_eq!(trunk.tracks().len(), 0, "nothing has ever set a track set");
4223 assert_eq!(trunk.track_generation(), 0);
4224 }
4225
4226 /// `set_tracks` is a **whole-set replacement**, not a merge/append, and
4227 /// bumps the generation by exactly one per call.
4228 ///
4229 /// MUTATION VERIFIED: changing `TrunkWriter::set_tracks`'s body to merge
4230 /// the old set with the new one (`let mut merged = state.tracks.to_vec();
4231 /// merged.extend(tracks); state.tracks = Arc::from(merged);`) instead of
4232 /// replacing outright makes the second `assert_eq!` below fail —
4233 /// `trunk.tracks()`'s track ids read back `[1, 7, 9]` (the old track 1
4234 /// still present) instead of `[7, 9]`. Recompiled and re-run to confirm
4235 /// the failure, then reverted.
4236 #[test]
4237 fn set_tracks_replaces_the_whole_set_and_bumps_generation_by_one_per_call() {
4238 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(8)));
4239 let writer = trunk.writer().unwrap();
4240
4241 writer.set_tracks(vec![opaque_track(1)]);
4242 assert_eq!(
4243 trunk
4244 .tracks()
4245 .iter()
4246 .map(|t| t.track_id)
4247 .collect::<Vec<_>>(),
4248 vec![1]
4249 );
4250 assert_eq!(trunk.track_generation(), 1);
4251
4252 // A completely different, larger set: if this were a merge/append
4253 // rather than a replacement, the old track_id 1 would still be
4254 // present alongside the two new ones.
4255 writer.set_tracks(vec![opaque_track(7), opaque_track(9)]);
4256 assert_eq!(
4257 trunk
4258 .tracks()
4259 .iter()
4260 .map(|t| t.track_id)
4261 .collect::<Vec<_>>(),
4262 vec![7, 9],
4263 "set_tracks must replace the set wholesale, not append to it"
4264 );
4265 assert_eq!(
4266 trunk.track_generation(),
4267 2,
4268 "generation must advance by exactly one per set_tracks call"
4269 );
4270 }
4271
4272 /// `track_generation` is stable across everything that is *not*
4273 /// `set_tracks` — publishing samples/events/segments/parts must never
4274 /// bump it, so a consumer polling the generation as a cheap "did the
4275 /// track set change" check cannot see false positives.
4276 ///
4277 /// MUTATION VERIFIED: adding `state.track_generation += 1;` to
4278 /// `TrunkWriter::publish` (simulating "generation accidentally bumped by
4279 /// unrelated activity") makes the final `assert_eq!` below fail — the
4280 /// generation reads back `11` (bumped once per one of the 10 published
4281 /// samples, on top of the `1` from `set_tracks`) instead of staying at
4282 /// `1`. Recompiled and re-run to confirm the failure, then reverted.
4283 #[test]
4284 fn generation_is_stable_across_unrelated_activity() {
4285 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(8)));
4286 let writer = trunk.writer().unwrap();
4287
4288 writer.set_tracks(vec![opaque_track(1)]);
4289 assert_eq!(trunk.track_generation(), 1);
4290
4291 for i in 0u8..10 {
4292 writer.publish(1, RetentionClass::Timed, sample(i, 4));
4293 }
4294 writer.publish_event(basic_event(1), EventAnchor::Media(MediaTime(0)));
4295
4296 assert_eq!(
4297 trunk.track_generation(),
4298 1,
4299 "publishing samples/events must never bump track_generation"
4300 );
4301 }
4302
4303 /// `set_tracks` wakes a registered [`Trunk::listen`] listener, the same
4304 /// broad `progress` channel [`SegmentWriter::publish_part`]/
4305 /// [`SegmentWriter::publish_segment`] already wake — see
4306 /// [`waiter_is_woken_when_the_awaited_part_lands_and_resolves_to_it`] for
4307 /// the identical pattern this test mirrors.
4308 ///
4309 /// MUTATION VERIFIED: removing `self.trunk.progress.notify(usize::MAX);`
4310 /// from `TrunkWriter::set_tracks` makes `listener.wait_deadline(..)`
4311 /// time out (`false`) instead of waking (`true`) within the 60s bound
4312 /// used below. Recompiled and re-run to confirm the failure, then
4313 /// reverted.
4314 #[test]
4315 fn set_tracks_wakes_a_registered_listener() {
4316 let trunk = Trunk::new(TrunkConfig::new(nz(4), nz(4), nz(4), nz(8), nz(8)));
4317 let writer = Arc::new(trunk.writer().unwrap());
4318
4319 // Register BEFORE the change — the documented no-missed-wakeup
4320 // ordering every other `listen()` test in this module follows.
4321 let listener = trunk.listen().expect("first registration must succeed");
4322
4323 let bg_writer = Arc::clone(&writer);
4324 let handle = thread::spawn(move || {
4325 thread::sleep(Duration::from_millis(50));
4326 bg_writer.set_tracks(vec![opaque_track(1)]);
4327 });
4328
4329 // Same generous, machine-independent bound as this module's other
4330 // wake tests — see their comments for why 60s asserts only "it woke
4331 // at all", not "it woke fast".
4332 let woken = listener.wait_deadline(std::time::Instant::now() + Duration::from_secs(60));
4333 assert!(woken, "listener must wake on set_tracks, not park forever");
4334 handle.join().unwrap();
4335
4336 assert_eq!(trunk.track_generation(), 1);
4337 }
4338}