<div align="center">
# Kenosis
**Production-grade ONNX model quantization. Zero Python. Single Native Binary.**
[](https://github.com/CoreEpoch/kenosis/actions)
[](LICENSE)
</div>
---
Kenosis is a Rust CLI toolkit for quantizing, validating, inspecting, and comparing ONNX models. Its flagship feature is **static INT8 quantization** (enabled by default) with fusion-aware QDQ placement that achieves up to **2.42× speedup** over FP32 baselines on stock ONNX Runtime, no custom operators required. Evaluated on **1,000 real-world images** from ImageNet-1K, Kenosis INT8 achieves **83.1–95.3% Top-1 predict agreement** with FP32 baselines across standard classifier architectures.
> **How it works:** [Fusion-Aware QDQ Placement: Achieving Native Kernel Fusion in ONNX via Graph Reordering](https://coreepoch.dev/research/kenosis-fusion-aware-qdq-placement.html)
## Benchmark Results
Kenosis quantizes the **PP-YOLOE+ object detection models**, an anchor-free architecture designed for efficient edge deployment:
| PP-YOLOE+ Small | 320×320 | 0.998 | **23ms** vs 44ms | **1.89×** | 7.9 MB (3.9× smaller) |
| PP-YOLOE+ Small | 416×416 | 0.998 | **43ms** vs 77ms | **1.80×** | 7.9 MB (3.9× smaller) |
| PP-YOLOE+ Small | 640×640 | 0.999 | **111ms** vs 187ms | **1.68×** | 7.9 MB (3.8× smaller) |
### Classifier Benchmarks (Kenosis PT/PC vs FP32 Baseline)
Fidelity evaluated on **1,000 images** from the ImageNet-1K validation set with model-specific preprocessing. All numbers correspond to the published evaluation in the companion paper.
| ResNet50 v2 | Kenosis (PT) | 0.980 | **94.8%** | **28.04ms** | **2.42×** | 30.6 MB (3.2× smaller) |
| ResNet50 v2 | Kenosis (PC) | 0.988 | **95.3%** | **28.02ms** | **2.42×** | 30.7 MB (3.2× smaller) |
| MobileNetV2 | Kenosis (PT) | 0.970 | **89.9%** | **5.22ms** | **1.33×** | 7.1 MB (1.9× smaller) |
| MobileNetV2 | Kenosis (PC) | 0.990 | **93.8%** | **5.23ms** | **1.33×** | 7.2 MB (1.8× smaller) |
| EfficientNet-Lite4 | Kenosis (PT) | 0.885 | **83.1%** | **19.39ms** | **1.41×** | 16.5 MB (3.0× smaller) |
| EfficientNet-Lite4 | Kenosis (PC) | 0.946 | **89.8%** | **19.44ms** | **1.41×** | 16.7 MB (3.0× smaller) |
> **Note:** SqueezeNet 1.1 and PP-YOLOE+ benchmarks are available in the [full paper](https://doi.org/10.5281/zenodo.20657990).
### Controlled Ablation: Fusion-Aware vs. Naive Placement
From the paper — identical weights and scales, differing only in QDQ placement:
| **ResNet50 v2** (PT) | **28.04ms** | 32.33ms (+15%) | **0.980** | 0.972 | **94.8%** | 93.0% |
| **MobileNetV2** (PT) | **5.22ms** | 7.77ms (+49%) | **0.970** | 0.954 | **89.9%** | 86.5% |
| **EfficientNet-Lite4** (PT) | **19.39ms** | 23.29ms (+20%) | **0.885** | 0.692 | **83.1%** | 62.8% |
| **ResNet50 v2** (PC) | **28.02ms** | 32.37ms (+16%) | **0.988** | 0.982 | **95.3%** | 95.0% |
| **MobileNetV2** (PC) | **5.23ms** | 7.89ms (+51%) | **0.990** | 0.976 | **93.8%** | 89.3% |
| **EfficientNet-Lite4** (PC) | **19.44ms** | 23.39ms (+20%) | **0.946** | 0.756 | **89.8%** | 70.2% |
> On MobileNetV2, naive placement produces a model **12% slower than FP32** (7.77ms vs 6.96ms). Fusion-aware placement restores a **1.33× speedup** with the same quantized weights.
## Key Features
| Static INT8 with ReLU-aware QDQ | ✅ | ❌ |
| Detection model mixed-precision | ✅ | ❌ |
| Non-vision tensor protection | ✅ | ❌ |
| Multi-input model calibration | ✅ | ❌ |
| Transformer & MatMul quantization | ✅ | ❌ |
| NLP synthetic calibration data | ✅ | ❌ |
| INT32 bias quantization w/ DQL | ✅ | ✅ |
| Per-channel weight quantization | ✅ | ✅ |
| Built-in validation + benchmarking | ✅ | ❌ |
| PaddlePaddle Constant extraction | ✅ | ❌ |
| Zero Python dependency | ✅ | ❌ |
| Cross-platform single binary | ✅ | ❌ |
## Install
```bash
cargo install kenosis-cli
```
Or build from source:
```bash
git clone https://github.com/CoreEpoch/kenosis.git
cd kenosis
cargo build --release
```
## Usage
### Static INT8 Quantization (default)
The primary quantization mode. Produces QDQ-format models that run on stock ONNX Runtime with full INT8 acceleration.
```bash
# Standard vision model (SqueezeNet, ResNet, EfficientNet, etc.)
kenosis quantize model.onnx -o model_int8.onnx
# Per-tensor weights (one scale per tensor, override the default per-channel mode)
kenosis quantize model.onnx -o model_int8.onnx --per-tensor
# PaddlePaddle models (PP-YOLOE+, PP-LCNet, etc.)
kenosis quantize ppyoloe.onnx -o ppyoloe_int8.onnx --extract-constants
# Custom calibration sample count
kenosis quantize model.onnx -o model_int8.onnx --n-calib 40
# External calibration data (raw f32 binary files)
kenosis quantize model.onnx -o model_int8.onnx --calib-dir ./calib_data/
```
### Validate Quantized Models
Compare a quantized model against its FP32 baseline — measures cosine similarity, Top-1 agreement, and latency side-by-side.
```bash
# Basic validation (default samples, 200 timed runs)
kenosis validate model.onnx model_int8.onnx
# Custom sample counts
kenosis validate model.onnx model_int8.onnx -n 500 --timed 500
```
Output:
```
════════════════════════════════════════════════════════
📊 Kenosis Validation Report
════════════════════════════════════════════════════════
▸ Cosine similarity: 0.999128 (min 0.9986)
▸ Top-1 agreement: 83/100 (83%)
▸ Latency: 2.82ms vs 6.03ms (2.13× speedup)
▸ Size: 1.24 MB vs 4.73 MB (3.8× smaller)
▸ Verdict: EXCELLENT — production ready
════════════════════════════════════════════════════════
```
### Inspect a Model
```bash
# Basic stats — ops, params, size, data types, largest tensors
kenosis inspect model.onnx
```
### Utility Commands
```bash
# Cast to FP16/BF16
kenosis cast model.onnx -o model_fp16.onnx --precision fp16
# Compare two models
kenosis diff model.onnx model_int8.onnx
```
## How Static INT8 Works
Kenosis's static INT8 pipeline applies seven coordinated optimizations:
1. **Self-calibration** — Automatically generates synthetic calibration inputs and runs them through the model via ONNX Runtime to collect per-tensor activation ranges. No external calibration data required. Multi-input models and NLP inputs (token IDs, attention masks) are handled automatically.
2. **Weight quantization** — INT8 symmetric per-tensor or per-channel. All scale computations in f64 to match ORT's internal precision.
3. **INT32 bias quantization** — `scale = activation_scale × weight_scale`, zero_point = 0. Wrapped with DequantizeLinear for ORT kernel fusion.
4. **Zero-point nudged activation quantization** — UINT8 asymmetric with post-hoc range adjustment ensuring `float 0.0` maps exactly to the quantized zero. Prevents rounding asymmetry from compounding across layers.
5. **Fusion-aware QDQ placement** — ORT's Python quantizer places QDQ nodes on every Conv/MatMul output independently. Kenosis detects `Conv/MatMul → Activation` pairs (ReLU, LeakyRelu, Clip, HardSwish, Sigmoid) at graph level and places QDQ *after* the activation instead. This gives ORT's runtime optimizer a cleaner pattern that fuses into a single INT8 kernel. Combined with second-pass wrapping of Add, Concat, MaxPool, and AveragePool, this maximizes QLinear fusions.
6. **Non-vision tensor protection** — For multi-input models (detection, segmentation), tensors reachable from non-primary inputs (scale_factor, image_shape) are traced through the graph and excluded from quantization. This prevents metadata paths from being crushed by INT8 range limits.
7. **Model output protection** — Tensors that are direct model outputs are never QDQ-wrapped, preserving full FP32 precision in detection head scores and bounding box coordinates.
## Detection Model Support
Kenosis handles the specific challenges of quantizing object detection models:
- **Multi-input calibration** — Auto-generates appropriate default values for secondary inputs (scale_factor → 1.0, shape tensors → 0.0)
- **PaddlePaddle weight handling** — Extracts inline Constant nodes, deduplicates shared weights (deepcopy tensors), and upgrades opset attributes (Squeeze, Unsqueeze, BatchNorm, Dropout)
- **Mixed-precision detection head** — Backbone and neck are fully INT8; detection head outputs and metadata paths stay FP32
- **Scale factor preservation** — The bounding box rescaling path remains live and dynamic, not frozen to calibration values
## Architecture
```
kenosis/
├── crates/
│ └── kenosis-core/ # Library: quantization engine
│ └── src/
│ ├── model.rs # OnnxModel load/save/traversal + Constant extraction
│ ├── static_int8.rs # Static INT8 QDQ quantization pipeline
│ ├── inspect.rs # Stats and analysis
│ ├── cast.rs # FP16/BF16 casting
│ ├── diff.rs # Model comparison
│ ├── proto.rs # ONNX protobuf type definitions
│ └── error.rs # Error types
├── apps/
│ └── kenosis-cli/ # Binary: CLI interface
│ └── src/commands/
│ ├── quantize.rs # quantize command (static INT8)
│ ├── validate.rs # validate command (accuracy + latency)
│ ├── inspect.rs # inspect command
│ ├── cast.rs # cast command
│ └── diff.rs # diff command
```
## License
Apache-2.0 — see [LICENSE](LICENSE). Redistribution must retain the [NOTICE](NOTICE) file per Apache License 2.0 §4(d).
## 📄 Cite This Work
If you use Kenosis or reference the fusion-aware QDQ placement method in your research or software, please cite the paper:
[](https://doi.org/10.5281/zenodo.20657990)
```bibtex
@article{coomler2026fusionaware,
author = {Coomler, Cory},
title = {Fusion-Aware QDQ Placement: Achieving Native Kernel Fusion in ONNX via Graph Reordering},
year = {2026},
publisher = {Zenodo},
doi = {10.5281/zenodo.20657990},
url = {https://doi.org/10.5281/zenodo.20657990}
}
```
See [CITATION.cff](CITATION.cff) for machine-readable citation metadata (APA, BibTeX, etc.).