moeflux 0.1.0-pre.3

Pure-Rust streaming-experts MoE inference on Metal. Forked from flash-moe; only the Metal kernels remain from upstream.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

//! Pure-Rust [moeflux](https://github.com/mdegans/moeflux) — streaming-experts
//! MoE inference on Metal.
//!
//! Forked from [flash-moe](https://github.com/danveloper/flash-moe);
//! the Metal kernels at `crates/moeflux/shaders/shaders.metal` retain
//! flash-moe's attribution. Everything else is original Rust.
//!
//! ## Platform
//!
//! macOS only. On non-macOS targets this crate exposes no symbols.
//!
//! ## Model variant
//!
//! Exactly one Cargo feature must select the compile-time model
//! shape:
//!
//! - `model-qwen3-5-a17b`
//! - `model-qwen3-6-35b-a3b`
//!
//! See `docs/model_variants.md` for the full shape table.
//!
//! ## Concurrency
//!
//! Single-threaded per [`Ctx`]. The Metal command queue is owned per
//! [`Ctx`], so two `Ctx`s can drive the GPU concurrently from
//! different threads (the Council reactor uses this for parallel
//! agent deliberation). [`Ctx`] is [`Send`] but not [`Sync`].
//!
//! ## Differential oracle
//!
//! moeflux is a pure-Rust port; the original C/Objective-C reference
//! implementation has been retired. The port is validated by
//! Rust-internal differential oracles — `tests/graph_diff_oracle.rs`
//! (the CPU backend vs. the Metal backend, per `Op`) and
//! `tests/batched_diff_oracle.rs` (batched vs. per-token paths), with
//! `tests/diff_oracle.rs` carrying the remaining Rust-internal checks.

#![cfg_attr(not(target_os = "macos"), no_std)]
#![deny(unsafe_op_in_unsafe_fn)]
#![warn(missing_docs)]

/// Pure-Rust port of the host-side dispatch.
#[cfg(target_os = "macos")]
pub mod riir;

/// Default backend re-export. The Rust port is the only path.
#[cfg(target_os = "macos")]
pub use riir::{
    CheckpointError, RsCtx as Ctx, RsError as Error, DEFAULT_MAX_CHECKPOINTS,
};