onde 0.1.0

On-device inference abstraction layer for SplitFire AI — wraps mistral.rs model management.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

# onde

**On-device inference abstraction layer for SplitFire AI.**

`onde` wraps [mistral.rs](https://github.com/EricLBuehler/mistral.rs) model management so that individual Tauri apps remain thin command wrappers. It handles:

- **Model metadata** — download sizes, display names, organisations, descriptions.
- **HuggingFace token resolution** — build-time literal or on-disk cache (required on iOS where the filesystem is sandboxed).
- **HuggingFace hub cache** — inspection, repair, and model download with a progress-callback API decoupled from Tauri.

## Platform support

| Platform | Backend |
|---|---|
| macOS | Metal-accelerated inference |
| iOS | Metal-accelerated inference |
| Android | CPU-only GGUF inference via candle |
| Windows | CPU-only baseline (CUDA opt-in via CI) |

## Usage

Add to your `Cargo.toml`:

```toml
[dependencies]
onde = "0.1.0"
```

Re-exports are available as `onde::mistralrs`, `onde::hf_hub`, and `onde::mistralrs_core` so downstream crates do not need direct dependencies on those crates.

## License

MIT