Crate ffai_argus 

Argus — FFai’s vision-language component

Named for Argus Panoptes, the all-seeing hundred-eyed watchman: image captioning, visual Q&A, and video understanding.

What runs

SmolVLM-256M-Instruct on candle — a SigLIP tower, a pixel-shuffle connector and a Llama text decoder, each gated tensor-by-tensor against the reference implementation (docs/plans/argus-launch-plan.md, steps 3-6). Video understanding composes ffai-media::stream_frames (rff-backed sampling) with per-frame captioning into a timed track.

mistral.rs is not rejected — it stays the documented path for the serving concerns it owns (quantized weights, grammar-constrained JSON), and the mistralrs-backend feature is reserved for it. It is not the path here because the version proven to serve SmolVLM is a git revision, and cargo publish refuses a git dependency — a constraint that has already made every downstream FFai crate unpublishable once.

Re-exports

pub use engine::SmolVlm;

Modules

clock

Instant on native, a zero clock on wasm.

cost

Re-export of ffai_core::cost.

decode

The decode loop: inputs_embeds -> generated tokens, greedy, on candle.

engine

The VlmEngine implementation: an ImageBuffer in, a caption out.

par

Parallel iterators on native, the identical serial ones on wasm.

preprocess

Image -> pixel_values: the content path between a decoded file and the vision tower.

prompt

Sequence assembly: turning an image + a question into the exact token sequence the model was trained on.

siglip

Our own SigLIP vision encoder — same maths, fewer passes over memory.

text

Our own SmolLM2 text tower — same maths, fewer passes over memory.

vision

SmolVLM’s vision tower and connector, on candle.

Functions

register

Install every Argus engine into a registry.