moeflux 0.1.0-pre.1

Pure-Rust streaming-experts MoE inference on Metal. Forked from flash-moe; only the Metal kernels remain from upstream.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

//! Pure-Rust [moeflux](https://github.com/mdegans/moeflux) — streaming-experts
//! MoE inference on Metal.
//!
//! Forked from [flash-moe](https://github.com/SuperEpic/flash-moe);
//! the Metal kernels at `crates/moeflux/shaders/shaders.metal` retain
//! flash-moe's attribution. Everything else is original Rust.
//!
//! ## Platform
//!
//! macOS only. On non-macOS targets this crate exposes no symbols.
//!
//! ## Model variant
//!
//! Exactly one Cargo feature must select the compile-time model
//! shape:
//!
//! - `model-qwen3-5-a17b`
//! - `model-qwen3-6-35b-a3b`
//!
//! See `docs/model_variants.md` for the full shape table.
//!
//! ## Concurrency
//!
//! Single-threaded per [`Ctx`]. The Metal command queue is owned per
//! [`Ctx`], so two `Ctx`s can drive the GPU concurrently from
//! different threads (the Council reactor uses this for parallel
//! agent deliberation). [`Ctx`] is [`Send`] but not [`Sync`].
//!
//! ## Differential oracle
//!
//! The original C-via-Objective-C implementation is wrapped as a
//! test-only helper at `tests/common/c_backend.rs`. moeflux's lib
//! ships only the Rust port; the C oracle exists solely to validate
//! the port via `tests/diff_oracle.rs`.

#![cfg_attr(not(target_os = "macos"), no_std)]
#![deny(unsafe_op_in_unsafe_fn)]
#![warn(missing_docs)]

/// Pure-Rust port of the host-side dispatch.
#[cfg(target_os = "macos")]
pub mod riir;

/// Default backend re-export. The Rust port is the only path.
#[cfg(target_os = "macos")]
pub use riir::{RsCtx as Ctx, RsError as Error};