Crate tensogram_core 

tensogram-core

This crate has been renamed to tensogram.

This package re-exports everything from tensogram for backwards compatibility. Update your dependency to use tensogram directly:

[dependencies]
tensogram = "0.15"

Modules

decode

dtype

encode

error

file

framing

hash

iter

Iterator types for lazy traversal of messages and objects.

metadata

pipeline

Shared encoding/filter/compression pipeline helpers for importers.

streaming

types

validate

Validation of tensogram messages and files.

wire

Structs

DataObjectDescriptor

Per-object descriptor — merges tensor metadata and encoding instructions.

DataPipeline

Encoding/filter/compression configuration for data objects.

DecodeOptions

Options for decoding.

EncodeOptions

Options for encoding.

FileIssue

A file-level issue (not tied to a specific message).

FileMessageIter

Lazy iterator over messages stored in a file.

FileValidationReport

Result of validating a .tgm file.

GlobalMetadata

Global message metadata (carried in header/footer metadata frames).

HashDescriptor

Hash descriptor for payload integrity verification.

HashFrame

Hash frame payload — per-object integrity hashes.

IndexFrame

Index frame payload — maps object ordinals to byte offsets.

MessageFlags

Flags in the message preamble indicating which optional frames are present.

MessageIter

Zero-copy iterator over messages in a byte buffer.

ObjectIter

Iterator over the decoded objects (tensors) in a single message.

StreamingEncoder

A streaming encoder that writes Tensogram frames progressively to a sink.

TensogramFile

A handle for reading/writing Tensogram message files.

ValidateOptions

Options passed to validate_message.

ValidationIssue

A single validation finding.

ValidationReport

Result of validating a single message.

Enums

ByteOrder

CompressionBackend

Selects which backend to use when both FFI and pure-Rust implementations are compiled in for the same codec (szip or zstd).

Dtype

FrameType

Frame type identifiers (uint16).

HashAlgorithm

IssueCode

Stable machine-readable issue codes.

IssueSeverity

Severity of a validation finding.

TensogramError

ValidationLevel

Validation levels, from lightest to most thorough.

Constants

DEFAULT_PARALLEL_THRESHOLD_BYTES

Default threshold below which the library runs sequentially even when threads > 0. Chosen to be well above the per-call rayon pool construction cost (~10 µs) but small enough not to starve encode paths that want parallelism.

ENV_THREADS

Env var consulted when the caller-provided threads is 0. Must parse as a u32; zero, missing, empty, or otherwise unparseable values all resolve to 0 (sequential execution).

RESERVED_KEY

Key reserved for library-managed metadata (ndim/shape/strides/dtype/provenance).

Functions

apply_pipeline

Apply a DataPipeline to a DataObjectDescriptor by setting its encoding / filter / compression fields and populating params.

compute_common

Extract keys common to ALL base entries.

compute_hash

Compute a hash of the given data, returning the hex-encoded digest.

decode

Decode all objects from a message buffer. Returns (global_metadata, list of (descriptor, decoded_data)).

decode_descriptors

Decode global metadata and per-object descriptors without decoding any payload data.

decode_metadata

Decode only global metadata from a message buffer, skipping payloads.

decode_object

Decode a single object by index (O(1) access via index frame). Returns (global_metadata, descriptor, decoded_data).

decode_range

Decode partial ranges from a data object.

decode_range_from_payload

encode

Encode a complete Tensogram message.

encode_pre_encoded

Encode a pre-encoded Tensogram message where callers supply already-encoded bytes.

messages

Create a zero-copy iterator over messages in a byte buffer.

objects

Create an iterator that decodes each object in a message on demand.

objects_metadata

Return an iterator over the DataObjectDescriptors in a message without decoding any payload data.

scan

Scan a multi-message buffer for message boundaries. Returns (offset, length) of each message found.

scan_file

Scan a file for message boundaries without loading the entire file into memory.

validate_buffer

Validate all messages in a byte buffer (may contain multiple messages).

validate_file

Validate all messages in a .tgm file.

validate_message

Validate a single message buffer.

verify_canonical_cbor

Verify that CBOR bytes are in RFC 8949 §4.2.1 canonical form.

verify_hash

Verify a hash descriptor against data.

Type Aliases

DecodedObject

A decoded object: its descriptor paired with its raw decoded payload bytes.

Result