Skip to main content

encode_pre_encoded

tensogram_core

Function encode_pre_encoded 

Source 
pub fn encode_pre_encoded(
    global_metadata: &GlobalMetadata,
    descriptors: &[(&DataObjectDescriptor, &[u8])],
    options: &EncodeOptions,
) -> Result<Vec<u8>, TensogramError>
Expand description

Encode a pre-encoded Tensogram message where callers supply already-encoded bytes.

Use this when the payload bytes have already been encoded/compressed by an external pipeline. The library will:

  • Validate object descriptors (shape, dtype, etc.)
  • Validate encoding/compression params via build_pipeline_config() (defense-in-depth)
  • Use the caller’s bytes directly as the encoded payload (no pipeline call)
  • Compute a fresh xxh3 hash over the caller’s bytes (overwrites any caller-supplied hash)
  • Preserve caller-supplied szip_block_offsets in descriptor params

Callers must NOT set:

  • emit_preceders = true — use StreamingEncoder::write_preceder() for streaming preceder support.
  • reject_nan = true or reject_inf = true — these flags are raw-input-only; pre-encoded bytes are opaque to the library and cannot be scanned. Setting either flag returns TensogramError::Encoding.