mnn-rs

Rust bindings for MNN (Mobile Neural Network), Alibaba's efficient and lightweight deep learning inference framework.

Features

• Safe Rust API: All MNN operations are wrapped in safe Rust types with proper error handling
• Cross-platform: Supports Windows, Linux, macOS, Android, and iOS
• Multiple Backends: CPU, CUDA, OpenCL, Vulkan, and Metal
• Static/Dynamic Linking: Choose between static or dynamic linking
• Async Support: Optional async API with Tokio integration
• Build from Source: Automatically clone and build MNN from GitHub

Quick Start

Default Build (Recommended)

By default, the crate will automatically clone and build MNN from GitHub:

cargo build

This requires:

• Git (for cloning MNN source)
• CMake (for building MNN)
• C++ compiler (MSVC on Windows, GCC/Clang on Linux/macOS)

Using Pre-built MNN

If you have a pre-built MNN library, you can disable the auto-build:

# Set MNN library paths
export MNN_LIB_DIR=/path/to/mnn/lib
export MNN_INCLUDE_DIR=/path/to/mnn/include

# Build without build-from-source feature
cargo build --no-default-features --features cpu,static

Usage

Basic Inference

use mnn_rs::{prelude::*, BackendType, ScheduleConfig};

fn main() -> Result<(), MnnError> {
    // Load model
    let interpreter = Interpreter::from_file("model.mnn")?;

    // Create session
    let config = ScheduleConfig::new()
        .backend(BackendType::CPU)
        .num_threads(4);
    let mut session = interpreter.create_session(config)?;

    // Get input tensor and write data
    let input = session.get_input(None)?;
    let input_data: Vec<f32> = vec![0.0; input.element_count() as usize];
    input.write(&input_data)?;

    // Run inference
    session.run()?;

    // Read output
    let output = session.get_output(None)?;
    let output_data: Vec<f32> = output.read()?;

    println!("Output: {:?}", &output_data[..10]);
    Ok(())
}

Async Inference (with Tokio)

use mnn_rs::{prelude::*, BackendType, ScheduleConfig};

#[tokio::main]
async fn main() -> Result<(), MnnError> {
    let interpreter = Interpreter::from_file("model.mnn")?;
    let config = ScheduleConfig::new().backend(BackendType::CPU);
    let mut session = interpreter.create_session(config)?;

    // Async inference
    session.run_async().await?;

    Ok(())
}

Features

Linking Mode

Backend Support

Precision Support

x86 SIMD Optimizations

Build Options

Async Support

Examples

See the examples/ directory for more usage examples:

• basic_inference.rs - Basic inference workflow
• async_inference.rs - Async inference with Tokio
• gpu_backend.rs - Using GPU backends

# Run basic inference example
cargo run --example basic_inference -- /path/to/model.mnn

# Run with build-from-source
cargo run --features build-from-source --example basic_inference -- /path/to/model.mnn

Cross-Compilation

Windows (x86_64-pc-windows-gnu)

# Install MinGW-w64 toolchain
rustup target add x86_64-pc-windows-gnu
cargo build --target x86_64-pc-windows-gnu --features build-from-source

Windows (i686-pc-windows-gnu)

rustup target add i686-pc-windows-gnu
cargo build --target i686-pc-windows-gnu --features build-from-source

Android

rustup target add aarch64-linux-android
export ANDROID_NDK_HOME=/path/to/ndk
cargo build --target aarch64-linux-android --features build-from-source

Environment Variables

API Documentation

See https://docs.rs/mnn-rs for full API documentation.

MNN Version

This crate is compatible with MNN 2.9.5+.

License

Licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT License (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution

Contributions are welcome! Please feel free to submit a Pull Request.

Acknowledgments

• MNN - Alibaba's Mobile Neural Network inference engine
• All contributors who helped with this project