onde 0.1.0

On-device inference abstraction layer for SplitFire AI — wraps mistral.rs model management.
Documentation

onde

On-device inference abstraction layer for SplitFire AI.

onde wraps mistral.rs model management so that individual Tauri apps remain thin command wrappers. It handles:

  • Model metadata — download sizes, display names, organisations, descriptions.
  • HuggingFace token resolution — build-time literal or on-disk cache (required on iOS where the filesystem is sandboxed).
  • HuggingFace hub cache — inspection, repair, and model download with a progress-callback API decoupled from Tauri.

Platform support

Platform Backend
macOS Metal-accelerated inference
iOS Metal-accelerated inference
Android CPU-only GGUF inference via candle
Windows CPU-only baseline (CUDA opt-in via CI)

Usage

Add to your Cargo.toml:

[dependencies]
onde = "0.1.0"

Re-exports are available as onde::mistralrs, onde::hf_hub, and onde::mistralrs_core so downstream crates do not need direct dependencies on those crates.

License

MIT