Struct StreamingEncoder 

pub struct StreamingEncoder<W: Write> { /* private fields */ }

A streaming encoder that writes Tensogram frames progressively to a sink.

Unlike crate::encode::encode, which builds the entire message in memory, StreamingEncoder writes each data object frame immediately. This allows encoding to a socket or pipe without buffering the full message.

The trade-off is that header-based index and hash frames are not possible; instead, these are written as footer frames when finish is called.

Example

use std::io::BufWriter;
use std::fs::File;
use tensogram::streaming::StreamingEncoder;
use tensogram::{GlobalMetadata, EncodeOptions};

let file = BufWriter::new(File::create("output.tgm").unwrap());
let meta = GlobalMetadata::default();
let mut enc = StreamingEncoder::new(file, &meta, &EncodeOptions::default()).unwrap();
// enc.write_object(&desc, &data).unwrap();
// enc.finish().unwrap();

Implementations

impl<W: Write> StreamingEncoder<W>

pub fn new( writer: W, global_meta: &GlobalMetadata, options: &EncodeOptions, ) -> Result<Self>

Begin a new streaming message.

Writes the preamble (with total_length = 0 for streaming mode) and a header metadata frame containing the global metadata.

pub fn write_preceder( &mut self, metadata: BTreeMap<String, Value>, ) -> Result<()>

Write a PrecederMetadata frame for the next data object.

The metadata map becomes base[0] in a GlobalMetadata CBOR wrapper. Must be followed by exactly one write_object or write_object_pre_encoded call before another write_preceder or finish.

pub fn write_object( &mut self, desc: &DataObjectDescriptor, data: &[u8], ) -> Result<()>

Encode and write a single data object frame.

The descriptor’s encoding/filter/compression pipeline is applied, the payload is hashed (if configured), and the frame is written immediately — no buffering.

When EncodeOptions.threads > 0 was passed to StreamingEncoder::new, the pipeline call may use up to that many threads internally (axis B). Axis A is not available in streaming mode — each write_object is a caller-paced event with no cross-object parallelism opportunity.

pub fn write_object_pre_encoded( &mut self, descriptor: &DataObjectDescriptor, pre_encoded_bytes: &[u8], ) -> Result<()>

Write a pre-encoded data object frame directly.

Unlike write_object, this method does not run the encoding pipeline — pre_encoded_bytes are written to the stream as-is. The descriptor must accurately describe the encoding that was already applied (encoding, filter, compression, params) so that decoders can reconstruct the original payload.

This method participates in the same preceder consumption logic as write_object and can be freely intermixed with it.

Errors

Returns an error if the descriptor is invalid or the frame cannot be written to the underlying writer.

pub fn finish(self) -> Result<W>

Finalize the streaming message.

Writes footer frames (payload metadata + hash + index) and the postamble. Consumes the encoder and returns the underlying writer.

pub fn object_count(&self) -> usize

Returns the number of data objects written so far.

pub fn bytes_written(&self) -> u64

Returns the total bytes written so far.

impl<W: Write + Seek> StreamingEncoder<W>

pub fn finish_with_backfill(self) -> Result<W>

Finalize the streaming message and back-fill the total_length field in both the preamble and postamble (v3 §7 and §9.4).

Equivalent to finish but at the end seeks back to the preamble offset (0) and the postamble’s total_length slot (end_pos - 16), writing the real message length to both. Readers can then backward-scan from EOF using the mirrored length without fallback to forward scanning.

Use this when the writer backs a seekable sink (file, cursor over a buffer) — it’s a no-op semantic change over finish() but enables O(1) backward scan on the produced file.

Errors

Returns any I/O error from the underlying writer’s seek, write_all, or flush.