RunNX
A minimal, mathematically verifiable ONNX runtime implementation in Rust.

Overview
Fast, fearless, and formally verified ONNX in Rust.
This project provides a minimal, educational ONNX runtime implementation focused on:
- Simplicity: Easy to understand and modify
- Verifiability: Formal mathematical verification using Why3 and property-based testing
- Performance: Efficient operations using ndarray
- Safety: Memory-safe Rust implementation with mathematical guarantees
Features
- ✅ Dual Format Support: JSON and binary ONNX protobuf formats
- ✅ Auto-detection: Automatic format detection based on file extension
- ✅ Graph Visualization: Beautiful terminal ASCII art and professional Graphviz export
- Terminal visualization with dynamic layout and rich formatting
- DOT format export for publication-quality diagrams (PNG, SVG, PDF)
- CLI integration with
--graphand--dotoptions - Topological sorting and cycle detection
- ✅ Basic tensor operations (
Add,Mul,MatMul,Conv,Relu,Sigmoid,Reshape,Transpose) - ✅ YOLO Model Support: Essential operators for YOLO object detection models
Concat: Tensor concatenation for feature fusionSlice: Tensor slicing operationsUpsample: Feature map upsampling for FPNMaxPool: Max pooling operationsSoftmax: Classification probability computationNonMaxSuppression: Object detection post-processing
- ✅ Formal mathematical specifications with Why3
- ✅ Property-based testing for mathematical correctness
- ✅ Runtime invariant verification
- ✅ Model loading and validation
- ✅ Inference execution
- ✅ Error handling and logging
- ✅ Benchmarking support
- ✅ Async support (optional)
- ✅ Command-line runner
- ✅ Comprehensive examples
Quick Start
Prerequisites
RunNX requires the Protocol Buffers compiler (protoc) to build:
# Ubuntu/Debian
# macOS
# Windows
Installation
Add this to your Cargo.toml:
[]
= "0.2.0"
Basic Usage
use ;
// Load a model (supports both JSON and ONNX binary formats)
let model = from_file?; // Auto-detects format
// Or explicitly:
// let model = Model::from_onnx_file("model.onnx")?; // Binary ONNX
// let model = Model::from_json_file("model.json")?; // JSON format
// Create input tensor
let input = from_array;
// Run inference
let outputs = model.run?;
// Get results
let result = outputs.get.unwrap;
println!;
Saving Models
use Model;
let model = /* ... create or load model ... */;
// Save in different formats
model.to_file?; // Auto-detects format from extension
model.to_onnx_file?; // Explicit binary ONNX format
model.to_json_file?; // Explicit JSON format
Command Line Usage
# Run inference on a model (supports both .onnx and .json files)
# Show model summary and graph visualization
# Generate Graphviz DOT file for professional diagrams
# Run with async support
Graph Visualization
RunNX provides comprehensive graph visualization capabilities to help you understand and debug ONNX model structures. You can visualize models both in the terminal and as publication-quality graphics.
Terminal Visualization
Display beautiful ASCII art representations of your model directly in the terminal:
# Show visual graph representation
# Show both model summary and graph
Example Output
Here's what the terminal visualization looks like for a complex neural network:
┌────────────────────────────────────────┐
│ GRAPH: neural_network_demo │
└────────────────────────────────────────┘
📥 INPUTS:
┌─ image_input [1 × 3 × 224 × 224] (float32)
┌─ mask_input [1 × 1 × 224 × 224] (float32)
⚙️ INITIALIZERS:
┌─ conv1_weight [64 × 3 × 7 × 7]
┌─ conv1_bias [64]
┌─ fc_weight [1000 × 512]
┌─ fc_bias [1000]
🔄 COMPUTATION FLOW:
│
├─ Step 1: conv1
│ ┌─ Operation: Conv
│ ├─ Inputs:
│ │ └─ image_input
│ │ └─ conv1_weight
│ │ └─ conv1_bias
│ ├─ Outputs:
│ │ └─ conv1_output
│ └─ Attributes:
│ └─ kernel_shape: [7, 7]
│ └─ strides: [2, 2]
│ └─ pads: [3, 3, 3, 3]
│
├─ Step 2: relu1
│ ┌─ Operation: Relu
│ ├─ Inputs:
│ │ └─ conv1_output
│ ├─ Outputs:
│ │ └─ relu1_output
│ └─ (no attributes)
[... more steps ...]
📤 OUTPUTS:
└─ classification [1 × 1000] (float32)
└─ segmentation [1 × 21 × 224 × 224] (float32)
📊 STATISTICS:
├─ Total nodes: 10
├─ Input tensors: 2
├─ Output tensors: 2
└─ Initializers: 4
🎯 OPERATION SUMMARY:
├─ Add: 1
├─ Conv: 2
├─ Flatten: 1
├─ GlobalAveragePool: 1
├─ MatMul: 1
├─ MaxPool: 1
├─ Mul: 1
├─ Relu: 1
└─ Upsample: 1
Graphviz Export
Generate professional diagrams using DOT format for Graphviz:
# Generate DOT file for Graphviz
# Convert to PNG (requires Graphviz installation)
# Convert to SVG for vector graphics
# Convert to PDF for documents
Example Graph Output
The DOT format generates clean, professional diagrams with:
- Green ellipses for input tensors
- Blue diamonds for initializers (weights/biases)
- Rectangular boxes for operations
- Red ellipses for output tensors
- Directed arrows showing data flow

Example: Multi-task neural network with classification and segmentation branches
DOT Format Output
The generated DOT file contains structured graph data that Graphviz uses to create the visualizations. Here's an excerpt of the DOT format:
digraph G
The DOT format uses:
- Nodes: Define graph elements with shapes, colors, and labels
- Edges: Define connections with
->arrows - Attributes: Control visual appearance and layout
- rankdir=TB: Top-to-bottom layout direction
For the complete DOT file example, see assets/complex_graph.dot.
Programmatic Usage
You can also generate visualizations programmatically:
use Model;
let model = from_file?;
// Print graph to terminal
model.print_graph;
// Generate DOT format
let dot_content = model.to_dot;
write?;
// The graph name box automatically adjusts to any length
// Works with short names like "CNN" or very long names like
// "SuperLongComplexNeuralNetworkGraphName"
Features
- Dynamic Layout: Graph title box automatically adjusts to accommodate any name length
- Topological Sorting: Shows correct execution order with dependency resolution
- Cycle Detection: Gracefully handles graphs with cycles
- Rich Information: Displays shapes, data types, attributes, and statistics
- Color Coding: Visual distinction between different node types in DOT format
- Multiple Formats: Terminal ASCII art and Graphviz-compatible DOT export
- Professional Quality: Publication-ready graphics for papers and presentations
Architecture
The runtime is organized into several key components:
Core Components
- Model: ONNX model representation and loading
- Graph: Computational graph with nodes and edges
- Tensor: N-dimensional array wrapper with type safety
- Operators: Implementation of ONNX operations
- Runtime: Execution engine with optimizations
File Format Support
RunNX supports both JSON and binary ONNX protobuf formats:
📄 JSON Format
- Human-readable: Easy to inspect and debug
- Text-based: Can be viewed and edited in any text editor
- Larger file size: More verbose due to text representation
- Extension:
.json
🔧 Binary ONNX Format
- Standard format: Official ONNX protobuf serialization
- Compact: Smaller file sizes due to binary encoding
- Interoperable: Compatible with other ONNX runtime implementations
- Extension:
.onnx
🎯 Auto-Detection
The Model::from_file() method automatically detects the format based on file extension:
.onnxfiles → Binary ONNX protobuf format.jsonfiles → JSON format- Other extensions → Attempts JSON parsing as fallback
For explicit control, use:
Model::from_onnx_file()for binary ONNX filesModel::from_json_file()for JSON files
Supported Operators
Basic Operators
| Operator | Status | Notes |
|---|---|---|
Add |
✅ | Element-wise addition |
Mul |
✅ | Element-wise multiplication |
MatMul |
✅ | Matrix multiplication |
Conv |
✅ | 2D Convolution |
Relu |
✅ | Rectified Linear Unit |
Sigmoid |
✅ | Sigmoid activation |
Reshape |
✅ | Tensor reshaping |
Transpose |
✅ | Tensor transposition |
YOLO-Specific Operators
| Operator | Status | Notes |
|---|---|---|
Concat |
✅ | Tensor concatenation for FPN |
Slice |
🚧 | Tensor slicing (simplified) |
Upsample |
🚧 | Feature upsampling (simplified) |
MaxPool |
🚧 | Max pooling (simplified) |
Softmax |
✅ | Classification probabilities |
NonMaxSuppression |
🚧 | NMS for detection (simplified) |
Legend: ✅ = Fully implemented, 🚧 = Simplified implementation, ❌ = Not implemented
Examples
Format Compatibility Demo
use *;
Simple Linear Model
use ;
use Array2;
Model Loading and Inference
# Format compatibility demonstration
# YOLO operator support demonstration
# YOLOv8n compatibility testing
# Format conversion between JSON and ONNX binary
# Simple model operations
# Formal verification examples
# Tensor operations
YOLO Model Support
RunNX now includes essential operators for YOLO-style object detection models:
use *;
Basic Model Loading
use ;
use HashMap;
Performance
The runtime includes benchmarking capabilities:
# Run benchmarks
# Generate HTML reports
Example benchmark results:
- Basic operations: ~10-50 µs
- Small model inference: ~100-500 µs
- Medium model inference: ~1-10 ms
Formal Verification
RunNX includes comprehensive formal verification capabilities to ensure mathematical correctness:
🔬 Mathematical Specifications
The runtime includes formal specifications for all tensor operations using Why3:
(** Addition operation specification *)
function add_spec (a b: tensor) : tensor
requires { valid_tensor a /\ valid_tensor b }
requires { a.shape = b.shape }
ensures { valid_tensor result }
ensures { result.shape = a.shape }
ensures { forall i. 0 <= i < length result.data ->
result.data[i] = a.data[i] + b.data[i] }
🧪 Property-Based Testing
Automatic verification of mathematical properties:
use ;
// Test addition commutativity: a + b = b + a
let result1 = tensor_a.add_with_contracts?;
let result2 = tensor_b.add_with_contracts?;
assert_eq!;
// Test ReLU idempotency: ReLU(ReLU(x)) = ReLU(x)
let relu_once = tensor.relu_with_contracts?;
let relu_twice = relu_once.relu_with_contracts?;
assert_eq!;
// Test Softmax probability distribution: sum = 1.0
let softmax_result = tensor.softmax_with_contracts?;
let sum: f32 = softmax_result.data.iter.sum;
assert!;
🔍 Runtime Verification
Dynamic checking of invariants during execution:
use InvariantMonitor;
let monitor = new;
let result = tensor.add?;
// Verify numerical stability and bounds
assert!;
🎯 Verified Properties
The formal verification system proves:
- Addition: Commutativity, associativity, identity
- Matrix Multiplication: Associativity, distributivity
- ReLU: Idempotency, monotonicity, non-negativity
- Sigmoid: Boundedness (0, 1), monotonicity, symmetry
- Numerical Stability: Overflow/underflow prevention
📊 Running Formal Verification
# Install Why3 (optional, for complete formal proofs)
# Run all verification (tests + proofs)
# Run only property-based tests (no Why3 required)
# Run verification example
# Generate verification report
Development
Quick Development with Justfile
RunNX includes a Justfile with convenient shortcuts for common development tasks:
# Install just command runner (one time setup)
# Show all available commands
# Quick development cycle
# Code quality
# Documentation
# Benchmarks
# Formal verification
# CI simulation
Alternatively, if you don't have just installed, use the included shell script:
# Show all available commands
# Quick development cycle
Running Tests
# Run all tests
# or with just
# Run tests with logging
RUST_LOG=debug
# Run specific test
Building Documentation
# Build and open documentation
# or with just
# Build with private items
Contributing
We welcome contributions! Please follow our development quality standards:
- Fork the repository
- Create a feature branch
- Make your changes following our Development QA Guidelines
- Add tests and documentation
- Run quality checks:
./scripts/quality-check.sh - Commit your changes (pre-commit hooks will run automatically)
- Submit a pull request
Development Quality Assurance
RunNX uses automated quality assurance tools to maintain code quality:
- Pre-commit hooks: Automatically run formatting, linting, and tests before each commit
- Code formatting: Consistent style enforced by
rustfmt - Linting: Comprehensive checks with
clippy(warnings treated as errors) - Comprehensive testing: Unit tests, integration tests, property-based tests, and doc tests
- Build verification: Ensures all code compiles successfully
For detailed information, see Development QA Guidelines.
To run quality checks manually:
# Run all quality checks with auto-fixes
# Or run individual checks
License
This project is licensed under
- Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)
Acknowledgments
- ONNX - Open Neural Network Exchange format
- ndarray - Rust's
ndarraylibrary - Candle - Inspiration for some design patterns
Roadmap
✅ Completed
- Dual Format Support: Both JSON and binary ONNX protobuf formats
- Auto-detection: Automatic format detection based on file extension
- Graph Visualization: Terminal ASCII art and professional Graphviz export
- Core Operators: Add, Mul, MatMul, Conv, ReLU, Sigmoid, Reshape, Transpose
- YOLO Operators: Concat, Slice, Upsample, MaxPool, Softmax, NonMaxSuppression
- Formal Verification: Mathematical specifications with Why3
- CLI Tool: Command-line runner with visualization capabilities
🚧 Planned
- Add more operators (BatchNorm, LayerNorm, etc.)
- GPU acceleration support
- Quantization support
- Model optimization passes
- WASM compilation target
- Python bindings
Documentation
📚 Additional Resources
- Release Notes - What's new in the latest version
- Complete Changelog - Full history of changes
- Release History - All previous release notes
- Contributing Guide - How to contribute to RunNX
- Development QA - Quality assurance guidelines
- Formal Verification - Mathematical verification details
🔗 External Links
- API Documentation - Complete API reference
- Crates.io - Package information
- GitHub Repository - Source code and issues
- CI/CD Status - Build and test results