Struct Sam3 

pub struct Sam3 {
    pub manifest: Manifest,
    pub store: Store,
    pub cfg: Config,
    pub tokenizer: Tokenizer,
    pub timing: Timing,
    /* private fields */
}

Fields

manifest: Manifest

store: Store

cfg: Config

tokenizer: Tokenizer

timing: Timing

Implementations

impl Sam3

pub fn detr_encoder( &mut self, fpn2: &[f32], text: &Text, ) -> Result<Vec<f32>, Error>

The DETR encoder: fpn2 [T, 256] (the 72 x 72 level, raster) and the prompt -> [T, 256].

pub fn detr_decoder( &mut self, enc: &[f32], text: &Text, ) -> Result<Decoded, Error>

The DETR decoder + scoring: enc [T, 256] and the prompt -> Decoded.

impl Sam3

pub fn mask_decoder( &mut self, hidden: &[f32], fpn: &[Vec<f32>; 3], enc: &[f32], text: &Text, ) -> Result<(Vec<f32>, Vec<f32>), Error>

(pred_masks [Q, S, S], semantic [S, S]) from the decoder’s last normalised query states, the FPN levels (channels-last) and the DETR encoder output.

impl Sam3

pub fn neck(&mut self, vit: &[f32]) -> Result<[Vec<f32>; 3], Error>

The backbone’s [T, 1024] (raster) -> FPN levels 0-2, channels-last f32: [288^2, 256], [144^2, 256], [72^2, 256].

impl Sam3

pub fn text(&self, ids: &[u32], mask: &[u32]) -> Result<Text, Error>

The CLIP text encoder over the tokens (ids, mask as the tokenizer gives them) -> the prompt features [L, 256].

impl Sam3

pub fn vit(&mut self, pixels: &[f32]) -> Result<Vec<f32>, Error>

pixels [3, S, S] -> the backbone’s last hidden state [T, 1024], raster order.

impl Sam3

pub fn load(dir: &Path) -> Result<Self, Error>

Loads the bundle, opens the NPU, loads every kernel and uploads the packed weights.

pub fn contexts(&self) -> (usize, usize)

Hardware contexts created and evicted so far (evictions happen when another process holds some of the NPU’s contexts).

pub fn segment(&mut self, pixels: &[f32], prompt: &str) -> Result<Output, Error>

Everything for one image and prompt: pixels is the preprocessed [3, 1008, 1008] image (preprocess). The text encoder (host) runs on a side thread while the ViT has the NPU.

pub fn forward(&mut self, pixels: &[f32], text: &Text) -> Result<Output, Error>

The forward from preprocessed pixels and an encoded prompt.