Struct Sam3 

pub struct Sam3 { /* private fields */ }

Implementations

impl Sam3

pub fn from_checkpoint( weights_path: &str, cfg: Sam3Config, ) -> Result<Sam3, Error>

Load a SAM3 checkpoint for native inference.

Load from .safetensors or .gguf (general.architecture = sam3).

For PyTorch checkpoints, convert with tests/sam3_parity_helpers/pt_to_safetensors.py or use community GGUF (e.g. rob-laz/sam3-gguf).

pub fn from_checkpoint_on( weights_path: &str, cfg: Sam3Config, device: Device, ) -> Result<Sam3, Error>

pub fn from_safetensors( weights_path: &str, cfg: Sam3Config, ) -> Result<Sam3, Error>

pub fn from_safetensors_on( weights_path: &str, cfg: Sam3Config, device: Device, ) -> Result<Sam3, Error>

pub fn compile_profile(&self) -> &CompileProfile

Tier-1 compile profile (sam.rlx.toml next to weights when present).

pub fn config(&self) -> &Sam3Config

pub fn tracker_weights(&self) -> &Sam3TrackerWeights

Returns the loaded tracker weights (used by the video basic test to confirm checkpoint coverage).

pub fn encoder_weights(&self) -> &Sam3EncoderWeights

pub fn decoder_weights(&self) -> &Sam3DecoderWeights

pub fn device(&self) -> Device

pub fn encode_image( &self, image_u8: &[u8], h_in: usize, w_in: usize, ) -> Result<Sam3EncodedImage, Error>

pub fn predict_image_text( &mut self, image_u8: &[u8], h_in: usize, w_in: usize, tokens: &[u32], ) -> Result<Sam3ImagePrediction, Error>

Run the vision trunk + 4-scale neck and return per-level [channels, h, w] feature maps with matching sinusoidal positional encodings. Used by the detector and as a parity gate. End-to-end image inference with a pre-tokenized text prompt (tokens has length seq_len == decoder context length, usually 32). Returns the same 3-tuple the public Sam3Processor.set_text_ prompt exposes — without NMS / score thresholding, which we leave to callers so parity tests can compare raw model outputs.

pub fn run_segmentation( &mut self, enc_memory_bf: &[f32], backbone_fpn: &[Vec<f32>], backbone_shapes: &[(usize, usize)], obj_queries_last_bf: &[f32], prompt_seq_first: &[f32], prompt_kpm: &[u8], batch: usize, enc_h: usize, enc_w: usize, num_queries: usize, seq_len: usize, ) -> Result<Sam3SegmentationOutput, Error>

Forward the segmentation head: cross-attend encoder memory to the text prompt, run the pixel decoder, and emit per-query mask logits plus the semantic mask.

pub fn run_dot_prod_scoring( &self, hs_bf: &[f32], prompt_seq_first: &[f32], prompt_kpm: &[u8], num_layers: usize, batch: usize, num_queries: usize, seq_len: usize, ) -> Result<Vec<f32>, Error>

Compute per-query, per-layer scores via mean-pooled text + linear projections + dot product.

pub fn run_decoder( &self, memory: &[f32], memory_pos: &[f32], memory_text: &[f32], text_attention_mask: &[u8], batch: usize, h: usize, w: usize, seq_len: usize, ) -> Result<Sam3DecoderOutput, Error>

Run the detector decoder. Inputs are the encoder memory in batch-first flat [batch, h*w, 256] plus matching positional encoding, and the text memory in seq-first [seq, batch, 256]. Returns intermediate layer outputs, refined boxes, and presence logits — the same triple the upstream model uses to derive scores and final box predictions.

pub fn run_encoder( &self, src_bchw: &[f32], src_pos_bchw: &[f32], prompt_seq_first: &[f32], prompt_kpm: &[u8], batch: usize, src_h: usize, src_w: usize, prompt_len: usize, ) -> Result<Vec<f32>, Error>

Run the detector encoder fusion on a single FPN level + text prompt. Returns the encoded image memory in batch-first flat [batch, h*w, 256].

pub fn encode_text_tokens( &self, tokens: &[u32], batch: usize, seq_len: usize, ) -> Result<Sam3TextEncoded, Error>

Run the text encoder on already-tokenized inputs. Returns the resized memory the detector consumes.

pub fn predict_neck( &mut self, image_u8: &[u8], h_in: usize, w_in: usize, ) -> Result<Vec<Sam3FeatureLevel>, Error>

pub fn patch_embed_image( &self, image_u8: &[u8], h_in: usize, w_in: usize, ) -> Result<Sam3EncodedImage, Error>

pub fn predict_image( &mut self, image_u8: &[u8], h_in: usize, w_in: usize, text_prompt: Option<&str>, boxes: Option<&[f32]>, points: Option<(&[f32], &[f32])>, ) -> Result<Sam3ImagePrediction, Error>

pub fn predict_video_frame( &mut self, state: &mut Sam3VideoState, image_u8: &[u8], h_in: usize, w_in: usize, text_prompt: Option<&str>, ) -> Result<Sam3VideoFramePrediction, Error>