Skip to main content

Crate taconite_bundle

Crate taconite_bundle 

Source
Expand description

Reads the bundles IRON’s exporters write for the Rust runtimes.

IRON compiles ahead of time: an exporter (iron/common/bundle.py has the writer side) builds every NPU kernel a model needs and writes a bundle directory a Rust runtime replays:

manifest.txt   one record a line: `<tag> <field>...`; a field `k=v` is
               also looked up by key. Blank lines and `#` lines are skipped.
tensors.txt    <name> <dtype> <d0,d1,...> <offset> <bytes>, one a line,
tensors.bin    into this blob (offsets 64-byte aligned; dtypes f32, bf16,
               u8, i32, little-endian)
kernels/       xclbins and instruction streams

Three manifest records mean the same thing in every bundle, and Manifest interprets them:

version <n>                        format version; must match the runtime's
param <name> <value...>            a model constant
xclbin <key> <file> <kernel name>  a hardware context kernels refer to by key

Every other record (kernels, steps, test cases) belongs to the model, and its runtime reads them from Manifest::records, in file order.

Structs§

Entry
Manifest
Record
One manifest line: a tag, then whitespace-separated fields.
Store
Every tensor of the bundle. On Unix tensors.bin is memory-mapped read-only: a tensor’s pages are read when it is first touched and stay reclaimable page cache, so loading a multi-GB bundle costs no heap and a runtime that uploads its weights and drops the store never holds them twice. Elsewhere (or if mapping fails) the file is read into a u64 buffer. Either way the 64-byte-aligned offsets tensors.txt records are aligned in memory (mappings are page-aligned), so the typed views below are sound. Don’t rewrite a bundle’s tensors.bin while a runtime has it loaded: exporters write new directories.
Xclbin
A hardware context: an xclbin and the kernel name inside it.

Enums§

DType
Error