cactus-rs

Safe Rust bindings for Cactus Compute's on-device engines: Needle 3 for tool calling, structured extraction and embeddings, and Whistle for speech-to-text.

What is Needle?

Needle 3 is a small model that reads a sentence and decides which of your functions to call, with which arguments. It runs on the CPU in about 100 MB of RAM at hundreds of tokens per second. This crate links the engine Cactus Compute publishes for it and gives it a typed Rust API.

What is Whistle?

Whistle is a 16.9 MB speech-to-text model that runs inside the same engine as Needle, so both live in one process. It transcribes clips of up to 30 seconds of 16 kHz mono audio in seven languages (en, de, fr, es, it, nl, pl), with word timestamps, keyword biasing and speech embeddings. Hand a clip to Needle and it transcribes and answers it in one call, so a spoken request becomes tool calls.

Quick Start

[dependencies]
cactus-rs = "0.2"
serde_json = "1.0"

use cactus_rs::needle::{Needle, Tool, Weights};
use serde_json::json;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let weights = Weights::fetch()?; // cached under ~/.cache/cactus-rs
    let mut needle = Needle::builder(weights)
        .system("You control the lights.")
        .tool(Tool::new(
            "set_light",
            "Turn a room light on or off",
            json!({
                "type": "object",
                "properties": { "room": { "type": "string" }, "on": { "type": "boolean" } },
                "required": ["room", "on"]
            }),
        ))
        .build()?;

    let completion = needle.complete("turn the kitchen light on")?;
    for call in completion.calls() {
        println!("{} {}", call.name(), call.arguments_json());
    }
    Ok(())
}

That prints set_light {"room":"kitchen","on":true}. The first build downloads the 1 MB engine library and the first run downloads the 35 MB weights; both are pinned and SHA-256 verified.

Speech

use cactus_rs::whistle::{Weights, Whistle};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut whistle = Whistle::builder(Weights::fetch()?).build()?; // 16.9 MB, cached
    let pcm: Vec<f32> = vec![0.0; 16_000]; // 16 kHz mono in [-1, 1], from your WAV reader
    let transcript = whistle.transcribe(&pcm)?;
    println!("{} ({})", transcript.text(), transcript.language_code());
    Ok(())
}

Once a Whistle has loaded the speech model, needle.complete_audio(&pcm) turns a clip into tool calls, with the transcript in completion.audio().

Features

• Tool calls as typed values, with call.parse::<YourStruct>()
• Structured extraction into your own types
• Text embeddings, 3072 dimensions
• Speech-to-text in seven languages, with language detection or forcing, keyword biasing and word timestamps
• Speech embeddings, 512 floats per 80 ms frame
• Voice to tool calls: Needle::complete_audio transcribes and answers in one engine call
• One model of each kind per process, enforced by the type system; Needle and Whistle are Send + Sync and share one engine lock
• Pinned, checksum-verified engine and weights, with fully offline builds
• cactus-sys for the raw C API underneath

Examples

Run one with cargo run -p cactus-rs --example lights.

Platforms

Intel macOS and Windows MSVC are not supported, because upstream ships no archive they can link. Offline builds, a local engine archive, a static C++ runtime and the other build options are covered in the cactus-sys documentation.

Roadmap

• The general Cactus engine (cactus_engine.h: chat, vision, streaming) as a second cactus-sys feature
• Streaming and microphone capture
• Clips longer than 30 s
• The grounding layer upstream implements in Python
• Tool-index persistence
• Subprocess isolation for several tuned models

Documentation

See https://docs.rs/cactus-rs

License

MIT OR Apache-2.0. The vendored needle.h and the smart-home test data are Cactus Compute's, under Apache-2.0, with provenance noted beside each. The spoken test clips were synthesised for this repository with Piper TTS from public-domain and CC0 voices, and the JFK clip is in the public domain; see cactus-rs/tests/data/README.md. This project is unofficial and not affiliated with Cactus Compute.