Kenosis

Production-grade ONNX model quantization. Zero Python. Single Native Binary.

Kenosis is a Rust CLI toolkit for quantizing, validating, inspecting, and comparing ONNX models. Its flagship feature is static INT8 quantization (enabled by default) with fusion-aware QDQ placement that achieves up to 2.46× speedup over FP32 baselines and 65% faster inference than the ORT Python quantizer — on stock ONNX Runtime, no custom operators required. Evaluated on 6,000 real-world images from ImageNet-1K and MS COCO val2017, Kenosis INT8 achieves 81.9–94.8% Top-1 predict agreement with FP32 baselines.

How it works: Fusion-Aware QDQ Placement: Achieving Native Kernel Fusion in ONNX via Graph Reordering

Benchmark Results

Kenosis quantizes the PP-YOLOE+ object detection models, an anchor-free architecture designed for efficient edge deployment:

Classifier Benchmarks (Kenosis INT8 vs FP32 Baseline)

Fidelity evaluated on 1,000 images from the ImageNet-1K validation set with model-specific preprocessing.

SqueezeNet cosine is measured under synthetic calibration inputs. Real-image accuracy metrics are omitted due to non-standard ONNX Zoo preprocessing.

Direct Comparison (Kenosis vs ORT Python Quantizer)

The ORT Python quantizer produces a SqueezeNet model that is slower than FP32 (8.13ms INT8 vs 6.60ms FP32) — broken Conv-ReLU fusion in action. Kenosis eliminates this regression entirely, delivering a 2.32× speedup over FP32.

Real-World Accuracy (Kenosis vs ORT Python)

Evaluated on 1,000 ImageNet-1K images (classifiers) and 5,000 MS COCO val2017 images (detector):

ORT's EfficientNet-Lite4 output cosine collapses to 0.364 — near-random outputs. Its MobileNetV2 achieves only 5.3% predict agreement with FP32. Kenosis maintains 81.9–94.8% agreement across all architectures.

Key Features

Install

cargo install kenosis-cli

Or build from source:

git clone https://github.com/CoreEpoch/kenosis.git
cd kenosis
cargo build --release

Usage

Static INT8 Quantization (default)

The primary quantization mode. Produces QDQ-format models that run on stock ONNX Runtime with full INT8 acceleration.

# Standard vision model (SqueezeNet, ResNet, EfficientNet, etc.)
kenosis quantize model.onnx -o model_int8.onnx

# Per-channel weights (better for models with high channel counts like ResNet)
kenosis quantize model.onnx -o model_int8.onnx --per-channel

# PaddlePaddle models (PP-YOLOE+, PP-LCNet, etc.)
kenosis quantize ppyoloe.onnx -o ppyoloe_int8.onnx --extract-constants

# Custom calibration sample count
kenosis quantize model.onnx -o model_int8.onnx --n-calib 40

# External calibration data (raw f32 binary files)
kenosis quantize model.onnx -o model_int8.onnx --calib-dir ./calib_data/

Validate Quantized Models

Compare a quantized model against its FP32 baseline — measures cosine similarity, Top-1 agreement, and latency side-by-side.

# Basic validation (default samples, 200 timed runs)
kenosis validate model.onnx model_int8.onnx

# Custom sample counts
kenosis validate model.onnx model_int8.onnx -n 500 --timed 500

Output:

  ════════════════════════════════════════════════════════
  📊  Kenosis Validation Report
  ════════════════════════════════════════════════════════
  ▸ Cosine similarity:  0.999128 (min 0.9986)
  ▸ Top-1 agreement:    83/100 (83%)
  ▸ Latency:            2.82ms vs 6.03ms (2.13× speedup)
  ▸ Size:               1.24 MB vs 4.73 MB (3.8× smaller)
  ▸ Verdict:             EXCELLENT — production ready
  ════════════════════════════════════════════════════════

Inspect a Model

# Basic stats — ops, params, size, data types, largest tensors
kenosis inspect model.onnx

Utility Commands

# Cast to FP16/BF16
kenosis cast model.onnx -o model_fp16.onnx --precision fp16

# Compare two models
kenosis diff model.onnx model_int8.onnx

How Static INT8 Works

Kenosis's static INT8 pipeline applies seven coordinated optimizations:

1. Self-calibration — Automatically generates synthetic calibration inputs and runs them through the model via ONNX Runtime to collect per-tensor activation ranges. No external calibration data required. Multi-input models and NLP inputs (token IDs, attention masks) are handled automatically.

2. Weight quantization — INT8 symmetric per-tensor or per-channel. All scale computations in f64 to match ORT's internal precision.

3. INT32 bias quantization — scale = activation_scale × weight_scale, zero_point = 0. Wrapped with DequantizeLinear for ORT kernel fusion.

4. Zero-point nudged activation quantization — UINT8 asymmetric with post-hoc range adjustment ensuring float 0.0 maps exactly to the quantized zero. Prevents rounding asymmetry from compounding across layers.

5. Fusion-aware QDQ placement — ORT's Python quantizer places QDQ nodes on every Conv/MatMul output independently. Kenosis detects Conv/MatMul → Activation pairs (ReLU, LeakyRelu, Clip, HardSwish, Sigmoid) at graph level and places QDQ after the activation instead. This gives ORT's runtime optimizer a cleaner pattern that fuses into a single INT8 kernel. Combined with second-pass wrapping of Add, Concat, MaxPool, and AveragePool, this maximizes QLinear fusions.

6. Non-vision tensor protection — For multi-input models (detection, segmentation), tensors reachable from non-primary inputs (scale_factor, image_shape) are traced through the graph and excluded from quantization. This prevents metadata paths from being crushed by INT8 range limits.

7. Model output protection — Tensors that are direct model outputs are never QDQ-wrapped, preserving full FP32 precision in detection head scores and bounding box coordinates.

Detection Model Support

Kenosis handles the specific challenges of quantizing object detection models:

• Multi-input calibration — Auto-generates appropriate default values for secondary inputs (scale_factor → 1.0, shape tensors → 0.0)
• PaddlePaddle weight handling — Extracts inline Constant nodes, deduplicates shared weights (deepcopy tensors), and upgrades opset attributes (Squeeze, Unsqueeze, BatchNorm, Dropout)
• Mixed-precision detection head — Backbone and neck are fully INT8; detection head outputs and metadata paths stay FP32
• Scale factor preservation — The bounding box rescaling path remains live and dynamic, not frozen to calibration values

Architecture

kenosis/
├── crates/
│   └── kenosis-core/           # Library: quantization engine
│       └── src/
│           ├── model.rs        # OnnxModel load/save/traversal + Constant extraction
│           ├── static_int8.rs  # Static INT8 QDQ quantization pipeline
│           ├── inspect.rs      # Stats and analysis
│           ├── cast.rs         # FP16/BF16 casting
│           ├── diff.rs         # Model comparison
│           ├── proto.rs        # ONNX protobuf type definitions
│           └── error.rs        # Error types
├── apps/
│   └── kenosis-cli/            # Binary: CLI interface
│       └── src/commands/
│           ├── quantize.rs     # quantize command (static INT8)
│           ├── validate.rs     # validate command (accuracy + latency)
│           ├── inspect.rs      # inspect command
│           ├── cast.rs         # cast command
│           └── diff.rs         # diff command

License

Apache-2.0 — see LICENSE.