Module framing 

Streaming record framing: the seam that cuts a decoded byte stream into records.

Framing is a transport- and format-agnostic seam. Any streaming or pull source (object-storage backfills in spate-s3, HTTP chunked bodies, WebSocket message streams, file tails) must turn a byte stream into records before a Deserializer decodes each one. That “one payload → one record” split lets a single decoder serve every source; the framer decides what one payload is.

spate-core owns the seam here (the RecordFramer trait, the FramerWriter decompressor shim, and the FramingContract handshake). The concrete framers are owned by the format crates, because how a byte stream splits into records is a property of the format rather than the transport. spate-json provides newline-delimited framing for NDJSON, a future spate-avro an object-container block framer, and so on. A source stays format-agnostic and receives its framer at pipeline-assembly time (e.g. S3Source::with_framer), so the same framer serves every streaming source and no source hard-codes a format.

A RecordFramer is fed decoded bytes in arbitrary chunks and yields completed record byte-slices. It must be a pure function of the byte stream, independent of how the bytes are chunked, so a source that resumes by record index replays deterministically; at-least-once resume depends on it (see the spate-s3 offset model). Compression is not part of framing. A source that decompresses wraps its decompressor around a FramerWriter, so the framer only ever sees already-decoded bytes.

Dispatch through Box<dyn RecordFramer> happens once per fed chunk, never per record, so each impl’s per-record scan stays monomorphized.

Structs

FramerWriter

A Write adapter over a RecordFramer, so a source whose decompressor (or any other Write-sink codec) emits decoded bytes through Write can feed a framer without the framer implementing Write itself. Every write forwards the buffer to RecordFramer::push.

Enums

FramingContract

What one payload emitted by a source represents, so the framework can pair a source with a deserializer without the two being coordinated by hand.

Traits

RecordFramer

A streaming record framer: fed decoded bytes, yields record byte-payloads.