Skip to main content

encode_pre_encoded

Function encode_pre_encoded 

Source
pub fn encode_pre_encoded(
    global_metadata: &GlobalMetadata,
    descriptors: &[(&DataObjectDescriptor, &[u8])],
    options: &EncodeOptions,
) -> Result<Vec<u8>>
Expand description

Encode a pre-encoded Tensogram message where callers supply already-encoded bytes.

Use this when the payload bytes have already been encoded/compressed by an external pipeline. The library will:

  • Validate object descriptors (shape, dtype, etc.)
  • Validate encoding/compression params via build_pipeline_config() (defense-in-depth)
  • Use the caller’s bytes directly as the encoded payload (no pipeline call)
  • Compute a fresh xxh3 hash over the caller’s bytes (overwrites any caller-supplied hash)
  • Preserve caller-supplied szip_block_offsets in descriptor params

Per-object preceder metadata is a streaming-mode concept (StreamingEncoder::write_preceder()); the buffered encode_pre_encoded path does not emit preceders.

Unlike encode(), this path does NOT run the finite-value check — the caller’s bytes are assumed to be already well-formed for the declared encoding and are written as-is. If the pre-encoded bytes decode to NaN / Inf, that round-trips through the wire unchanged.