cera-cli

Command-line interface for the cera LLM inference engine. Installs a cera binary for running, chatting with, inspecting, and benchmarking GGUF / LeapBundles models locally.

Note: In version 0.6.0, Cera will introduce breaking API changes to simplify usage and consolidate several APIs across the engine and language bindings. Follow updates in Releases.

Note: Part of a learning-experiment project exploring LLM inference internals in Rust, see the project README. Not intended for production use.

Install

cargo install cera-cli

This builds the cera binary. For an Apple Metal or wgpu GPU build:

cargo install cera-cli --features metal   # or gpu

Usage

Point at a local model (a .gguf file, a .json LeapBundles manifest, or a directory containing one) or let it auto-download a bundle by id/quant from huggingface.co/LiquidAI/LeapBundles (cached under $HOME/.cache/cera). Supported architectures: lfm2, lfm2moe (routed mixture-of-experts), qwen2/qwen3, llama (incl. classic Mistral), and granite; see the cera README for the full list and modality support.

# Generate from a local GGUF
cera run --model model.gguf --prompt "Explain quantization in one sentence."

# Auto-download a bundle and generate
cera run --bundle-id LFM2.5-1.2B-Instruct --quant Q4_0 --prompt "Hello"

# Constrain output to valid JSON (bundled grammar) or a custom GBNF
cera run -m model.gguf -p "List 3 colors as JSON" --json
cera run -m model.gguf -p "..." --grammar @schema.gbnf

# Tool calling: pass tool schemas (inline JSON or @file). stdout gets ONLY the
# JSON array of calls (the assistant reply + timing stream to stderr), so it
# pipes cleanly. Add --constrain-tools to force a valid call.
cera run -m model.gguf -p "Weather in Paris?" --tools @tools.json | jq .
cera run -m model.gguf -p "Weather in Paris?" --tools @tools.json --constrain-tools

# Interactive multi-turn chat REPL (keeps the prefix cache warm across turns)
cera chat --bundle-id LFM2.5-1.2B-Instruct --quant Q4_0

# No model source at all, on a terminal, without --no-tui: pick one from the
# published catalog. Type to filter, Enter to choose a bundle then its
# quantization, Esc to back out.
cera chat

# Attach a LoRA adapter (llama.cpp .gguf or PEFT .safetensors) for the session
cera run -m model.gguf -p "..." --lora adapter.safetensors
cera chat -m model.gguf --lora adapter.gguf

# Extract hidden-state embeddings for a prompt (mean-pooled [hidden_size] vector)
cera embed -m model.gguf -p "a chunk of text to embed"
cera embed -m model.gguf -p "a chunk" --per-token   # full [T*hidden_size] matrix, one row per token
cera embed -m model.gguf -p "a chunk" --json        # JSON array output instead of space-separated floats

Commands

# Run Voice Activity Detection on a WAV audio file
cera vad --model models/silero_vad.gguf --audio test.wav --threshold 0.5 --json

# Run directly from any Hugging Face model repository
cera run --hf LiquidAI/LFM2.5-1.2B-Instruct-GGUF --quant Q4_0 --prompt "Hello"

Run cera <command> --help for the full flag list. Common run flags: --max-tokens (default 256), --temperature (default 0.7), --device (cpu / gpu / metal / auto, default auto), --grammar / --json, and --lora to attach a LoRA adapter. For tool calling, --tools <JSON|@file> passes an array of OpenAI-style function schemas (rendered into the chat template; the reply's tool calls are parsed to a JSON array on stdout), and --constrain-tools (requires --tools) forces a well-formed, correctly-typed call via a grammar + lazy trigger. run, chat, and embed all accept --lora <PATH> (a llama.cpp .gguf or PEFT .safetensors adapter); it applies to every forward pass, generation and hidden-state extraction alike. For a PEFT .safetensors adapter whose alpha differs from its rank, pass --lora-alpha <ALPHA> (scale = alpha / rank; .gguf adapters carry alpha in their metadata).

CPU tuning

The CPU backend auto-detects thread count and core affinity per device. To pin them for benchmarking or tuning, set CERA_DECODE_THREADS=<n> (fixed decode width), CERA_PREFILL_THREADS=<n> (prefill width on its own, so the two can be swept independently), CERA_THREADS=<n> (override the detected perf-core count), or CERA_PIN=0 (disable affinity, which also lifts the clamp the other two are subject to). CERA_POOL_STATS=1 annotates each run with how many dispatches silently fell back to running serially, which is worth having on for any width sweep. See the crate README for the full list. On Android, bench also prints the device's thermal headroom per run (a sustained CPU benchmark heats the SoC within seconds, so a raw tok/s figure otherwise conflates the change under test with thermal drift).

License

Apache-2.0 OR MIT.