Expand description
Sliding-window JSON publishing over moq-net tracks.
A window is an ordered run of records the publisher appends to the back of and drops from the
front of. Unlike stream, which preserves a log forever in one group, and
snapshot, which keeps only the latest value, a window keeps a bounded
stretch of records and lets a reader join it at any point.
§Why a log can’t do this
The obvious alternative is an append-only log that rolls its group and re-seeds the new one with the records it still holds. That breaks the reader: re-seeded records are indistinguishable from new ones, so a reader that was keeping up receives them twice. This mode exists to make the restatement explicit, so a reader can tell “you already have these” from “here is another one”.
§On the wire
The first frame of every group names the retained offset and a decodable records suffix. Its
optional start identifies the suffix when a checkpoint bound omits older records. Later frames
are tagged push and pop ops, positional against the group header.
Indices stop at 2^53 - 1, the largest integer represented exactly by both implementations.
Trimming is therefore an op, not a group boundary. Dropping a record costs one small frame inside the shared compression window instead of a roll that would throw that window away.
§Group boundaries are invisible
The publisher rolls a group when the ops in it outgrow
ProducerConfig::op_ratio times the header that opened it, exactly as
snapshot rolls on its delta budget. That is purely a compression decision:
there is no caller-driven cut and no age bound, and a Consumer never surfaces it. A header
restating records a reader already has yields nothing, so however often the publisher rolls, the
reader sees one continuous stream of Events. ProducerConfig::checkpoint_records bounds
the suffix repeated on each roll for a long-lived window.
§What a reader is told
A reader gets Event::Push when a record arrives, Event::Pop when a contiguous range
leaves, and Event::Skip when a range was dropped before this reader saw it. A reader that
keeps up sees pushes and pops; one that falls a group behind learns from the header’s offset which
records it will never get, rather than silently missing them.
§Choosing a layer
Producer and Consumer own a track. Encoder and Decoder are the same logic
without it, for when something else is already in charge of the track; the encoder owns the
retained window and says where the group boundaries fall, and the decoder turns frames into
events.
Structs§
- Consumer
- Consumes a sliding window of JSON records from a track, yielding one event per change.
- Consumer
Config - Configuration for a
Decoder, and so for theConsumerwrapping one. - Decoder
- Reconstructs window events from frame payloads.
- Encoded
- One encoded frame, and the group boundary it implies.
- Encoder
- Encodes window edits into frame payloads, deciding where the group boundaries fall.
- Group
- Decodes one MoQ group’s frames while borrowing the continuous window state.
- Pending
- An encoded frame the caller has not yet acknowledged writing.
- Producer
- Publishes a sliding window of JSON records over a track.
- Producer
Config - Configuration for an
Encoderand theProducerwrapping one.
Enums§
- Event
- One change to the window, as the consumer sees it.