Crate taconite_sam3 

SAM3 (Segment Anything 3) text-prompted instance segmentation on an AMD XDNA NPU.

The bundle iron/applications/sam3/export_sam3.py writes holds every compiled IRON kernel and the model’s weights (NPU ones pre-packed); this crate replays the forward the Python app (iron/applications/sam3) runs, stage for stage:

stage	NPU	host (here)
CLIP text encoder		all of it (text.rs)
ViT backbone, 32 layers	all of it: patch embed, Linears, RoPE, attention, GELU, residual adds + LayerNorms	the first LayerNorm, once (vit.rs)
FPN neck	ConvTs, 1x1s, 3x3s	GELU, pixel shuffles (neck.rs)
DETR encoder, 6 layers	projections, self-attention, folded prompt cross-attention, MLP	LayerNorms, prompt softmax (detr.rs)
DETR decoder, 6 layers	all six layers’ vision keys/values (one GEMM)	the 201-query layers, box refinement, scoring (detr.rs)
mask decoder	pixel-decoder 3x3s, folded mask head, prompt cross-attention	GroupNorms, upsampling (mask.rs)

Sam3::segment is the whole thing; the stage methods are public so sam3 check can test each on the bundle’s reference inputs.

Re-exports

pub use post::Instance;

pub use post::instances;

pub use post::preprocess;

Modules

bundle

The bundle iron/applications/sam3/export_sam3.py writes: manifest.txt (kernels, model constants, tests), tensors.txt + tensors.bin (every weight and reference tensor), kernels/, the tokenizer files and test cases. See that script’s docstring for the format; the records every IRON bundle shares (version, param, xclbin) and the tensor store are taconite-bundle’s.

cpu

Host-side f32 math: threading helpers, linear layers, LayerNorm, the activations and multi-head attention. Everything is written so the inner loops vectorise (8 independent accumulators, no branches) and spread over scoped threads.

npu

The NPU side: every kernel of the bundle loaded once (kernels naming the same xclbin share its hardware context – 13 in all, under NPU2’s 16), and the device buffers they run over.

pack

flm.GEMM’s B packing (iron/operators/flm/packing.py), for the weights built per prompt (the folded cross-attentions and the mask head): a row-major [K, N] matrix reordered into the order the compute tiles read it and quantized to bfp16ebs8 (8 values along k share an exponent, 9 bytes a block), optionally with a per-column bias chunk in front of each column-block. Checked byte for byte against the Python packer by sam3 check (pack_test.*).

post

Pre- and post-processing, as HF’s Sam3ImageProcessor:

tokenizer

CLIP’s byte-level BPE tokenizer (HF CLIPTokenizerFast), zero-dependency.

Structs

Config

Model constants (from the manifest’s param lines).

Decoded

The decoder’s outputs (and per-layer values for checking).

Output

The model’s raw outputs, as Sam3Model returns them (batch of one).

Sam3

Text

An encoded prompt: feats [L, 256] (L = 32), valid [L].

Timing

Wall time per stage, in first-seen order; NPU dispatch time is kept under npu:<kernel>.

Enums

Error