Skip to main content

EncodeOptions

Struct EncodeOptions 

Source
pub struct EncodeOptions {
    pub hash_algorithm: Option<HashAlgorithm>,
    pub emit_preceders: bool,
    pub compression_backend: CompressionBackend,
    pub threads: u32,
    pub parallel_threshold_bytes: Option<usize>,
    pub reject_nan: bool,
    pub reject_inf: bool,
}
Expand description

Options for encoding.

Fields§

§hash_algorithm: Option<HashAlgorithm>

Hash algorithm to use for payload integrity. None = no hashing.

§emit_preceders: bool

Reserved for future buffered-mode preceder support.

Currently, setting this to true in buffered mode (encode()) returns an error — use StreamingEncoder::write_preceder instead. The streaming encoder ignores this field; it emits preceders only when write_preceder() is called explicitly.

§compression_backend: CompressionBackend

Which backend to use for szip / zstd when both FFI and pure-Rust implementations are compiled in.

Defaults to Ffi on native (faster, battle-tested) and Pure on wasm32 (FFI cannot exist). Override with TENSOGRAM_COMPRESSION_BACKEND=pure env variable, or set this field explicitly.

§threads: u32

Thread budget for the multi-threaded coding pipeline.

  • 0 (default) — sequential (current behaviour). Can be overridden at runtime via TENSOGRAM_THREADS=N.
  • 1 — explicit single-threaded execution (bypasses env).
  • N ≥ 2 — scoped pool of N workers. Output bytes are byte-identical to the sequential path regardless of N.

When more than one data object is being encoded the budget is spent axis-B-first (intra-codec parallelism) — this codebase tends to have a small number of very large messages. See the multi-threaded pipeline guide for the full policy.

Ignored with a one-time tracing::warn! when the threads cargo feature is disabled.

§parallel_threshold_bytes: Option<usize>

Minimum total payload bytes below which the parallel path is skipped even when threads > 0.

None uses crate::DEFAULT_PARALLEL_THRESHOLD_BYTES (64 KiB). Set to Some(0) to force the parallel path for testing; set to Some(usize::MAX) to force sequential.

§reject_nan: bool

Reject NaN values in float payloads before the encoding pipeline runs. Default false (backwards-compatible).

When true, raw input for float dtypes (float16/bfloat16/float32/float64/complex64/complex128) is scanned element-by-element. On first NaN the encode fails with TensogramError::Encoding carrying the element index and dtype. Integer and bitmask dtypes are skipped (zero cost).

Applies before any pipeline stage, so the contract is pipeline-independent: callers get the same guarantee whether they pick encoding="none", "simple_packing", or a lossy float compressor. This is the primary user-visible mitigation for the corners documented in plans/RESEARCH_NAN_HANDLING.md §3.1 (silent Inf corruption in simple_packing) and §3.3 (undefined NaN behaviour in zfp/sz3).

Setting this flag on encode_pre_encoded or StreamingEncoder::write_object_pre_encoded returns TensogramError::Encoding — pre-encoded bytes are opaque to the library and cannot be meaningfully scanned. Callers who want the strict contract must use encode or StreamingEncoder::write_object on raw input.

Parallel scans (when threads > 0 and input exceeds the parallel threshold) short-circuit per worker, so the reported element index may not be the globally first NaN. Pass threads = 0 for deterministic first-index reporting.

§reject_inf: bool

Reject +Inf / -Inf values in float payloads before the encoding pipeline runs. Default false (backwards-compatible).

Same semantics as reject_nan but for infinities. The error message includes the sign (+Inf or -Inf) alongside the element index and dtype.

Closes the silent-corruption gotcha documented in plans/RESEARCH_NAN_HANDLING.md §3.1: simple_packing::compute_params accepts Inf input and produces numerically-useless parameters (binary_scale_factor = i32::MAX) that silently decode to NaN everywhere. Turning this flag on catches the problem at encode time.

Trait Implementations§

Source§

impl Clone for EncodeOptions

Source§

fn clone(&self) -> EncodeOptions

Returns a duplicate of the value. Read more
1.0.0 · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for EncodeOptions

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter. Read more
Source§

impl Default for EncodeOptions

Source§

fn default() -> EncodeOptions

Returns the “default value” for a type. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> Pointable for T

Source§

const ALIGN: usize

The alignment of pointer.
Source§

type Init = T

The type for initializers.
Source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more