pub struct EncodeOptions {
pub hash_algorithm: Option<HashAlgorithm>,
pub emit_preceders: bool,
pub compression_backend: CompressionBackend,
pub threads: u32,
pub parallel_threshold_bytes: Option<usize>,
pub reject_nan: bool,
pub reject_inf: bool,
}Expand description
Options for encoding.
Fields§
§hash_algorithm: Option<HashAlgorithm>Hash algorithm to use for payload integrity. None = no hashing.
emit_preceders: boolReserved for future buffered-mode preceder support.
Currently, setting this to true in buffered mode (encode())
returns an error — use StreamingEncoder::write_preceder instead.
The streaming encoder ignores this field; it emits preceders only
when write_preceder() is called explicitly.
compression_backend: CompressionBackendWhich backend to use for szip / zstd when both FFI and pure-Rust implementations are compiled in.
Defaults to Ffi on native (faster, battle-tested) and Pure on
wasm32 (FFI cannot exist). Override with
TENSOGRAM_COMPRESSION_BACKEND=pure env variable, or set this
field explicitly.
threads: u32Thread budget for the multi-threaded coding pipeline.
0(default) — sequential (current behaviour). Can be overridden at runtime viaTENSOGRAM_THREADS=N.1— explicit single-threaded execution (bypasses env).N ≥ 2— scoped pool ofNworkers. Output bytes are byte-identical to the sequential path regardless ofN.
When more than one data object is being encoded the budget is spent axis-B-first (intra-codec parallelism) — this codebase tends to have a small number of very large messages. See the multi-threaded pipeline guide for the full policy.
Ignored with a one-time tracing::warn! when the threads
cargo feature is disabled.
parallel_threshold_bytes: Option<usize>Minimum total payload bytes below which the parallel path is
skipped even when threads > 0.
None uses crate::DEFAULT_PARALLEL_THRESHOLD_BYTES (64 KiB).
Set to Some(0) to force the parallel path for testing; set to
Some(usize::MAX) to force sequential.
reject_nan: boolReject NaN values in float payloads before the encoding pipeline
runs. Default false (backwards-compatible).
When true, raw input for float dtypes
(float16/bfloat16/float32/float64/complex64/complex128)
is scanned element-by-element. On first NaN the encode fails
with TensogramError::Encoding carrying the element index and
dtype. Integer and bitmask dtypes are skipped (zero cost).
Applies before any pipeline stage, so the contract is
pipeline-independent: callers get the same guarantee whether
they pick encoding="none", "simple_packing", or a lossy
float compressor. This is the primary user-visible mitigation
for the corners documented in plans/RESEARCH_NAN_HANDLING.md
§3.1 (silent Inf corruption in simple_packing) and §3.3
(undefined NaN behaviour in zfp/sz3).
Setting this flag on encode_pre_encoded or
StreamingEncoder::write_object_pre_encoded
returns TensogramError::Encoding — pre-encoded bytes are
opaque to the library and cannot be meaningfully scanned.
Callers who want the strict contract must use encode or
StreamingEncoder::write_object
on raw input.
Parallel scans (when threads > 0 and input exceeds the
parallel threshold) short-circuit per worker, so the reported
element index may not be the globally first NaN. Pass
threads = 0 for deterministic first-index reporting.
reject_inf: boolReject +Inf / -Inf values in float payloads before the
encoding pipeline runs. Default false (backwards-compatible).
Same semantics as reject_nan but for
infinities. The error message includes the sign (+Inf or
-Inf) alongside the element index and dtype.
Closes the silent-corruption gotcha documented in
plans/RESEARCH_NAN_HANDLING.md §3.1: simple_packing::compute_params
accepts Inf input and produces numerically-useless parameters
(binary_scale_factor = i32::MAX) that silently decode to NaN
everywhere. Turning this flag on catches the problem at encode
time.
Trait Implementations§
Source§impl Clone for EncodeOptions
impl Clone for EncodeOptions
Source§fn clone(&self) -> EncodeOptions
fn clone(&self) -> EncodeOptions
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Debug for EncodeOptions
impl Debug for EncodeOptions
Source§impl Default for EncodeOptions
impl Default for EncodeOptions
Source§fn default() -> EncodeOptions
fn default() -> EncodeOptions
Auto Trait Implementations§
impl Freeze for EncodeOptions
impl RefUnwindSafe for EncodeOptions
impl Send for EncodeOptions
impl Sync for EncodeOptions
impl Unpin for EncodeOptions
impl UnsafeUnpin for EncodeOptions
impl UnwindSafe for EncodeOptions
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more