Crate ferrox_inference 

Ferrox — a pure-Rust GGUF / MoE inference engine.

This crate is a facade. It contains no logic of its own: it re-exports the workspace under one name so a dependent writes one line in Cargo.toml instead of six, and so the project is findable on crates.io (the name ferrox belongs to an unrelated crate).

The command-line tools are not here. cargo install ferrox-cli installs the ferrox binary; ferrox-server is the OpenAI-compatible HTTP server. Shipping a second binary called ferrox from this crate would just fight the first one over ~/.cargo/bin.

Layout

The stack, bottom to top:

Module	Crate	What it is
gguf	ferrox-gguf	GGUF mmap reader, sharded checkpoints
quant	ferrox-quant	Block layouts and fused dequant+dot
safetensors	ferrox-safetensors	SafeTensors mmap reader
core	ferrox-core	Tensor ops, RoPE, GQA, KV cache
moe	ferrox-moe	Expert routing and dispatch
models	ferrox-models	Loaders and decoder stacks
[api]	ferrox-api	Route constants + wire DTOs (feature api)

Example

use ferrox_inference::gguf::ShardedGguf;

let file = ShardedGguf::open("model.gguf")?;
println!("{} tensors", file.tensor_count());

Features

• metal — Apple Metal kernels. Apple Silicon only.
• cuda — CUDA/NVRTC kernels. Needs a CUDA toolkit at build time. Held to “must compile”: there is no pinned benchmark host and no published receipts for it. See docs/FEATURES.md.
• api — pull in ferrox-api for client-side route constants.

Neither GPU feature is on by default, because both are wrong to assume: metal does not build off Apple Silicon and cuda needs a toolkit that most machines do not have.

Re-exports

pub use ferrox_core as core;

pub use ferrox_gguf as gguf;

pub use ferrox_models as models;

pub use ferrox_moe as moe;

pub use ferrox_quant as quant;

pub use ferrox_safetensors as safetensors;

Constants

VERSION

The workspace version this facade was built from.