Skip to main content

Crate ztensor_compat

Crate ztensor_compat 

Source
Expand description

Foreign tensor formats, projected into the zTensor object model — and the layout profiles that assemble an object once it is read.

Every format here builds a Catalog of names, shapes, types and byte locations, then hands back an ordinary Source. There is no per-format type in the public API, because after projection there is nothing per-format left to say:

let src = ztensor_compat::open("model.safetensors")?;
let t = src.tensor("layer.weight")?;
let bytes = t.data()?.map()?;  // zero-copy where the file allows it

Projections are read-only and honest. A part’s payload says which of the three shapes it has: a raw addressable range, something stored under an encoding, or bytes only the format’s own reader can produce. It also Caps is a direct reading of that. Nothing is silently reinterpreted or dequantized, and nothing degrades quietly to a copy.

A projected file reports its actual offsets; the .zt guarantees (the 4096 floor, page exclusivity, digests) hold only for .zt files. To upgrade a foreign checkpoint, convert it: Writer::ingest copies any source into a canonical .zt file.

Modules§

csr
Assembling a zt.sparse_csr/1 object.

Structs§

Open
How to open. Mirrors ztensor::read::Options, minus shard resolution: no foreign format has a shard table.

Constants§

FORMATS
Every label detect can return.

Functions§

detect
Sniffs the format of a tensor file. Returns one of FORMATS.
index
Opens without mapping: metadata and addresses only.
index_all
Indexes several files as one name space, mapping none of them.
open
Opens a tensor file of any supported format.
open_all
Opens several files as one name space. See Open::open_all.
options
Opening options: a vocabulary to read with, and whether to map.