Crate tensogram_ffi 

Tensogram C FFI

Exposes the tensogram library to C and C++ callers via opaque handles, typed accessor functions, and a flat C ABI.

Memory ownership rules:

• Handles returned by tgm_* functions are owned by the caller. Free them with the matching tgm_*_free function.
• Pointers returned by accessor functions (e.g. tgm_object_shape) are borrowed from the handle and valid until the handle is freed.
• tgm_bytes_t returned by encode functions must be freed with tgm_bytes_free.

JSON schema for tgm_encode

The metadata_json argument to tgm_encode is a JSON object with:

• "descriptors" (array, required): one entry per data object. Each entry merges tensor info and encoding pipeline info into a single object: type, ndim, shape, strides, dtype, byte_order, encoding, filter, compression. Additional keys are stored as params.
• "base" (array, optional): per-object application metadata.
• Any other top-level keys (e.g. "mars") are stored under the message-level _extra_ map. The CBOR metadata frame is free-form — the wire-format version lives exclusively in the preamble (see plans/WIRE_FORMAT.md §3) and must NOT be supplied by callers. A legacy "version" top-level field is tolerated for pre-0.17 schema compatibility and silently discarded.

Structs

TgmBufferIter

Opaque handle for iterating over messages in a byte buffer.

TgmBytes

An owned byte buffer returned by encode functions.

TgmDecodeMaskOptions

Decode-side companion to TgmEncodeMaskOptions. Pass a pointer to opt out of canonical NaN / Inf restoration. Pass NULL for the default restore_non_finite = true.

TgmEncodeMaskOptions

Mask-companion options for encode entry points (see plans/WIRE_FORMAT.md §6.5 and docs/src/guide/nan-inf-handling.md). Pass a pointer to this struct to opt into NaN / ±Inf substitution with bitmask companion frames.

TgmFile

File handle.

TgmFileIter

Opaque handle for iterating over messages in a file.

TgmMessage

Decoded message: global metadata + decoded (descriptor, payload) pairs.

TgmMetadata

Metadata-only handle (no decoded payloads).

TgmObjectIter

Opaque handle for iterating over objects within a single message.

TgmScanEntry

Scan result: array of (offset, length) pairs.

TgmScanResult

Opaque handle for scan results.

TgmStreamingEncoder

Opaque handle for a streaming encoder that writes data objects progressively.

Enums

TgmError

Constants

TGM_WIRE_VERSION

Wire-format version emitted and required by this build of the library.

Functions

tgm_buffer_iter_count

Return the total number of messages in the buffer iterator.

tgm_buffer_iter_create

Create a buffer message iterator.

tgm_buffer_iter_free

Free a buffer iterator handle.

tgm_buffer_iter_next

Advance the buffer iterator. On success, sets out_buf and out_len to the next message slice (borrowed from the original buffer).

tgm_bytes_free

Free a byte buffer returned by tgm_encode.

tgm_compute_hash

Compute a hash of the given data. Returns TGM_ERROR_OK on success, fills out with a tgm_bytes_t containing the hex-encoded hash string (NOT null-terminated). Free with tgm_bytes_free.

tgm_decode

Decode a complete message (global metadata + all object payloads).

tgm_decode_metadata

Decode only the global metadata (no payload bytes are read).

tgm_decode_object

Decode a single object by index.

tgm_decode_range

Decode partial ranges from a data object.

tgm_decode_with_options

Decode with explicit NaN / Inf restoration options.

tgm_doctor_to_json

Run environment diagnostics and serialise the report as a JSON byte buffer.

tgm_encode

Encode a Tensogram message from JSON metadata and raw data slices.

tgm_encode_pre_encoded

Encode a Tensogram message from JSON metadata and pre-encoded payload bytes.

tgm_encode_with_options

Encode with explicit NaN / Inf mask-companion options.

tgm_error_string

Convert an error code to a human-readable string. Returns a static string (always valid, never NULL).

tgm_file_append

Encode and append a message to the file. Same JSON schema as tgm_encode for metadata_json.

tgm_file_append_raw

Append raw message bytes to the file.

tgm_file_append_with_options

Append a message to a file with explicit NaN / Inf mask-companion options.

tgm_file_close

Close a file handle and release resources.

tgm_file_create

Create a new Tensogram file for writing.

tgm_file_decode_message

Decode message at index from the file. On success fills out with a TgmMessage handle.

tgm_file_iter_create

Create a file message iterator from an open TgmFile.

tgm_file_iter_free

Free a file iterator handle.

tgm_file_iter_next

Advance the file iterator. On success, fills out with a TgmBytes buffer containing the raw message bytes (caller owns, free with tgm_bytes_free).

tgm_file_message_count

Count messages in the file (may trigger lazy scan).

tgm_file_open

Open an existing Tensogram file for reading.

tgm_file_path

Returns the file path as a null-terminated string. The pointer is valid until the file handle is closed.

tgm_file_read_message

Read raw message bytes at index. On success fills out with a TgmBytes buffer.

tgm_last_error

Returns a pointer to the last error message, or NULL if no error. The pointer is valid until the next FFI call on the same thread.

tgm_message_free

Free a decoded message handle.

tgm_message_metadata

Extract a metadata handle from a decoded message. The metadata handle is independent — free it separately with tgm_metadata_free.

tgm_message_num_decoded

Returns the number of decoded payload buffers. Equivalent to tgm_message_num_objects — kept for ABI compatibility.

tgm_message_num_objects

Returns the number of decoded objects in this message handle. For tgm_decode this equals the total object count; for tgm_decode_object this is always 1.

tgm_message_version

Returns the wire format version the decoder read from the preamble.

tgm_metadata_free

Free a metadata handle.

tgm_metadata_get_float

Look up a float value by dot-notation key.

tgm_metadata_get_int

Look up an integer value by dot-notation key. Returns default_val if the key is not found or is not an integer.

tgm_metadata_get_string

Look up a string value by dot-notation key (e.g. “mars.class”). Returns NULL if the key is not found or is not a string. The pointer is valid until the metadata handle is freed.

tgm_metadata_num_objects

Returns the number of objects described in the global metadata.

tgm_metadata_version

Returns the wire format version.

tgm_object_byte_order

Returns the byte order string (“big” or “little”). Valid until message freed.

tgm_object_compression

Returns the compression string (e.g. “none”, “zstd”). Valid until message freed.

tgm_object_data

Returns a pointer to the decoded payload bytes for a decoded object. decoded_index is the index into the decoded objects array (0 for the first decoded object, regardless of the original object index). out_len receives the byte length.

tgm_object_dtype

Returns the dtype as a null-terminated string (e.g. “float32”). The pointer is valid until the message is freed.

tgm_object_filter

Returns the filter string (e.g. “none”, “shuffle”). Valid until message freed.

tgm_object_hash_type

Returns the hash type string (“xxh3”) or NULL if no hash. Valid until message freed.

tgm_object_hash_value

Returns the hash value hex string or NULL if no hash. Valid until message freed.

tgm_object_iter_create

Create an object iterator from raw message bytes.

tgm_object_iter_free

Free an object iterator handle.

tgm_object_iter_next

Advance the object iterator. On success, fills out with a TgmMessage handle containing exactly one decoded object (the next in sequence).

tgm_object_ndim

Returns the number of dimensions for object at index.

tgm_object_shape

Returns a pointer to the shape array. Length is tgm_object_ndim(). The pointer is valid until the message is freed.

tgm_object_strides

Returns a pointer to the strides array. Length is tgm_object_ndim().

tgm_object_type

Returns the object type string (e.g. “ndarray”). Valid until message freed.

tgm_payload_encoding

Returns the encoding string for a data object descriptor (e.g. “none”, “simple_packing”). The pointer is valid until the message is freed.

tgm_payload_has_hash

Returns 1 if the i-th data object has a populated inline hash slot, 0 otherwise.

tgm_scan

Scan a buffer for message boundaries.

tgm_scan_count

Returns the number of messages found by tgm_scan.

tgm_scan_entry

tgm_scan_free

Free a scan result handle.

tgm_simple_packing_compute_params

Compute simple_packing parameters for a set of f64 values.

tgm_streaming_encoder_count

Return the number of objects written so far.

tgm_streaming_encoder_create

Create a streaming encoder writing to a file.

tgm_streaming_encoder_create_with_options

Streaming-encoder constructor with NaN / Inf mask-companion options.

tgm_streaming_encoder_finish

Finalize the streaming encoder, writing footer and closing the file.

tgm_streaming_encoder_free

Free a streaming encoder without finalizing (abandons the output).

tgm_streaming_encoder_write

Write a single data object to the streaming encoder.

tgm_streaming_encoder_write_pre_encoded

Write a single pre-encoded data object to the streaming encoder.

tgm_streaming_encoder_write_preceder

Write a PrecederMetadata frame for the next data object.

tgm_validate

Validate a single Tensogram message buffer.

tgm_validate_file

Validate all messages in a .tgm file.