ffai-core

Shared types, engine traits, and the registry for FFai — the AI media toolkit, remade with Rust.

This is the crate every other FFai crate depends on and no FFai crate depends on. It holds no models and no algorithms: it defines the shapes everything else speaks in.

The idea

FFai is built the way ffmpeg is built — one trait per task, many engines per trait, selected by name. AsrEngine is the AVCodec of speech; --engine whisper-candle is -c:v libx264.

use ffai_core::engine::{AsrEngine, AsrOptions};
use ffai_core::registry::EngineRegistry;

let mut reg = EngineRegistry::new();
ffai_mercury::register(&mut reg);

let engine = reg.asr(Some("whisper-candle"))?;   // or None for the default
let transcript = engine.transcribe(&audio, &AsrOptions::default())?;

What's in here

Candle is re-exported as ffai_core::candle, so every engine shares one Tensor and one Device and buffers move between models without conversion.

fastmath / fastops — why they live here

candle's CPU backend evaluates tanhf/erf per element on one core; its rayon use covers conv2d and nothing else. On the shape a SigLIP MLP actually runs, (1, 1024, 3072), candle's .gelu() took 44.01 ms and the kernel here 1.22 ms — with the caption it feeds byte-identical to the reference.

They live in ffai-core because three engines had independently written their own range-reduced exp, and the three had drifted in exactly the line that decides whether the win happens: one left an f32::round in the loop and one an f32::floor, both of which are libm calls that keep the loop scalar — so two of the three had removed the call and put an equivalent barrier straight back. One module with the oracle tests is what stops that happening a fourth time.

Two things they are careful about: gelu_erf and gelu_tanh are different functions (they differ by ~1e-3, and a test asserts they disagree so a refactor cannot alias them), and tanh switches to a Maclaurin series below |x| = 0.02 because the 1 - 2/(e^{2x}+1) form catastrophically cancels there.

cost — a win is a counter that went down

A loaded box swings more than most optimizations are worth, so verdicts that can be deterministic are. cost counts the work rather than timing it: same input, same number, on any machine under any load. It also separates scalar from vectorised transcendentals, because conflating them (they are ~36x apart) once produced a cost model that predicted 32 s against a 16 s measurement.

Two conventions worth knowing

EngineStatus is honest. A registered engine is Stub, Experimental, or Stable, and Stable means it has been gated against a reference implementation — not that it works. ffai engines prints it.

Absent is not empty. Transcript::words and Transcript::speakers are Option<Vec<_>>. None means the stage was not requested; Some(vec![]) means it ran and found nothing. Collapsing those into a bare Vec would make a skipped stage indistinguishable from a stage that found nothing.

License

MIT OR Apache-2.0. Model weights carry their own licenses, surfaced at selection time by ffai-models.