mnn-rs

Rust bindings for MNN (Mobile Neural Network), Alibaba's efficient and lightweight deep learning inference framework.

Features

• Safe Rust API: All MNN operations are wrapped in safe Rust types with proper error handling
• Cross-platform: Supports Windows, Linux, macOS, Android, and iOS
• Prebuilt Binaries: Automatically download prebuilt binaries - no CMake or C++ compiler required!
• Multiple Backends: CPU, CUDA, OpenCL, Vulkan, and Metal
• Static/Dynamic Linking: Choose between static or dynamic linking
• Async Support: Optional async API with Tokio integration
• Build from Source: Option to build MNN locally when needed

Quick Start

Default Build (Recommended)

By default, the crate will automatically download prebuilt MNN binaries from GitHub Releases:

cargo build

No prerequisites required! The prebuilt binaries are available for:

• Windows (x86_64 MSVC, x86 MSVC)
• Linux (x86_64, aarch64)
• macOS (x86_64 Intel, aarch64 Apple Silicon)
• Android (arm64-v8a, armeabi-v7a)
• iOS (arm64 device, arm64 simulator)

Custom Prebuilt URL

You can specify a custom download URL for prebuilt binaries:

export MNN_PREBUILT_URL=https://your-server.com/mnn-x86_64-pc-windows-msvc.tar.gz
cargo build

Building from Source

If you need to build MNN locally (e.g., for custom build options):

cargo build --features build-from-source --no-default-features

This requires:

• Git (for cloning MNN source)
• CMake (for building MNN)
• C++ compiler (MSVC on Windows, GCC/Clang on Linux/macOS)

Using Pre-built MNN

If you have a pre-built MNN library, you can use it directly:

# Set MNN library paths
export MNN_LIB_DIR=/path/to/mnn/lib
export MNN_INCLUDE_DIR=/path/to/mnn/include

# Build without auto-download
cargo build --no-default-features --features cpu,static

Usage

Basic Inference

use mnn_rs::{prelude::*, BackendType, ScheduleConfig};

fn main() -> Result<(), MnnError> {
    // Load model
    let interpreter = Interpreter::from_file("model.mnn")?;

    // Create session
    let config = ScheduleConfig::new()
        .backend(BackendType::CPU)
        .num_threads(4);
    let mut session = interpreter.create_session(config)?;

    // Get input tensor and write data
    let input = session.get_input(None)?;
    let input_data: Vec<f32> = vec![0.0; input.element_count() as usize];
    input.write(&input_data)?;

    // Run inference
    session.run()?;

    // Read output
    let output = session.get_output(None)?;
    let output_data: Vec<f32> = output.read()?;

    println!("Output: {:?}", &output_data[..10]);
    Ok(())
}

Async Inference (with Tokio)

use mnn_rs::{prelude::*, BackendType, ScheduleConfig};

#[tokio::main]
async fn main() -> Result<(), MnnError> {
    let interpreter = Interpreter::from_file("model.mnn")?;
    let config = ScheduleConfig::new().backend(BackendType::CPU);
    let mut session = interpreter.create_session(config)?;

    // Async inference
    session.run_async().await?;

    Ok(())
}

Features

Linking Mode

Backend Support

Precision Support

x86 SIMD Optimizations

Build Options

Async Support

Examples

See the examples/ directory for more usage examples:

• basic_inference.rs - Basic inference workflow
• async_inference.rs - Async inference with Tokio
• gpu_backend.rs - Using GPU backends

# Run basic inference example
cargo run --example basic_inference -- /path/to/model.mnn

Cross-Compilation

Prebuilt binaries are available for most cross-compilation targets. If prebuilt binaries are not available for your target, enable build-from-source:

cargo build --target aarch64-linux-android --features build-from-source --no-default-features

Android

# Install Android target
rustup target add aarch64-linux-android

# Build for Android (uses prebuilt binaries by default)
cargo build --target aarch64-linux-android

# Or build from source (requires NDK and Ninja)
export ANDROID_NDK_HOME=/path/to/android-ndk
cargo build --target aarch64-linux-android --features build-from-source --no-default-features

Supported Android targets:

• aarch64-linux-android (arm64-v8a)
• armv7-linux-androideabi (armeabi-v7a)
• x86_64-linux-android
• i686-linux-android

Building from source requirements:

• Android NDK (set ANDROID_NDK_HOME or NDK_HOME environment variable)
• Ninja build system (choco install ninja on Windows, brew install ninja on macOS)

iOS

# Install iOS target
rustup target add aarch64-apple-ios
rustup target add aarch64-apple-ios-sim

# Build for iOS device (uses prebuilt binaries by default)
cargo build --target aarch64-apple-ios

# Build for iOS simulator (Apple Silicon Macs)
cargo build --target aarch64-apple-ios-sim

# Build for iOS simulator (Intel Macs)
cargo build --target x86_64-apple-ios

Supported iOS targets:

• aarch64-apple-ios (iOS device)
• aarch64-apple-ios-sim (iOS simulator on Apple Silicon)
• x86_64-apple-ios (iOS simulator on Intel Macs)

Linux

# Standard build
cargo build --target x86_64-unknown-linux-gnu

# Cross-compile for ARM
rustup target add aarch64-unknown-linux-gnu
cargo build --target aarch64-unknown-linux-gnu

macOS

# Intel Macs
cargo build --target x86_64-apple-darwin

# Apple Silicon Macs
cargo build --target aarch64-apple-darwin

Windows

# MSVC (recommended)
cargo build --target x86_64-pc-windows-msvc

# MinGW
rustup target add x86_64-pc-windows-gnu
cargo build --target x86_64-pc-windows-gnu

Environment Variables

API Documentation

See https://docs.rs/mnn-rs for full API documentation.

MNN Version

This crate is compatible with MNN 2.9.5+.

License

Licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT License (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution

Contributions are welcome! Please feel free to submit a Pull Request.

Acknowledgments

• MNN - Alibaba's Mobile Neural Network inference engine
• All contributors who helped with this project