burn-bitnet 0.1.0

BitNet quantization family for Burn — b1.58 ternary weights + v2 Hadamard activations
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

# burn-bitnet — BitNet Quantization for Burn

[![CI](https://github.com/sehaxe/burn-bitnet/actions/workflows/ci.yml/badge.svg)](https://github.com/sehaxe/burn-bitnet/actions/workflows/ci.yml)
[![Crates.io](https://img.shields.io/crates/v/burn-bitnet)](https://crates.io/crates/burn-bitnet)
[![License: AGPL-3.0](https://img.shields.io/badge/license-AGPL--3.0-blue.svg)](LICENSE)
[![Burn](https://img.shields.io/badge/Burn-0.21-orange.svg)](https://burn.dev)

**BitNet quantization family** for the [Burn](https://burn.dev) deep learning
framework. 1.58-bit ternary weight quantization + BitNet v2 Hadamard-based
activation quantization. Straight-through estimator (STE) gradients throughout.

> Papers: [BitNet b1.58](https://arxiv.org/abs/2402.17764) (Ma et al., 2024),
> [BitNet v2](https://arxiv.org/abs/2504.18415) (Wang et al., 2025).

## Install

```bash
cargo add burn-bitnet
```

## Quick start

```rust
use burn_bitnet::{weight_quant_ternary, quantize_4bit, quantize_8bit};

// Quantize weights to ternary {-scale, 0, +scale}
let w_q = weight_quant_ternary(weight);

// Quantize activations to 4-bit with Hadamard
let x_q = quantize_4bit(activation);

// 8-bit with Hadamard
let x_q = quantize_8bit(activation);
```

## API

| Function | Reference | What | Input |
|----------|-----------|------|-------|
| `weight_quant_ternary` | b1.58 (2024) | `W → sign(W−μ)·scale` | `[M, N]` |
| `activation_quant_8bit` | b1.58 (2024) | absmax per-token → int8 | `[B, T, D]` |
| `bitnet_v2_quantize` | v2 (2025) | Hadamard + absmax/absmean | `[B, T, D]` |
| `fast_walsh_hadamard` | v2 (2025) | FWHT O(n log n) | `[M, N]` |
| `quantize_4bit` | v2 (2025) | `bitnet_v2_quantize(x, 4)` | `[B, T, D]` |
| `quantize_8bit` | v2 (2025) | `bitnet_v2_quantize(x, 8)` | `[B, T, D]` |

## How it works

```
Ternary weight quantization (b1.58):
    scale = mean(|W|)
    W_q = sign(W − mean(W)) · scale

Tensor shapes: [in_features, out_features]

BitNet v2 quantization:
    x → FWHT → quantize → FWHT → output

8-bit: absmax per-token → int8 range [-128, 127]
4-bit: absmean per-token → int4 range [-8, 7]
```
    
All functions use the straight-through estimator (STE): forward pass uses
quantized values, backward gradient flows through the unquantized input.

## License

AGPL-3.0. See [LICENSE](LICENSE).