# rusty_esp_dsp
[](https://github.com/remade-with-rust) [](https://www.mata.network) [](https://crates.io/crates/rusty_esp_dsp) [](https://docs.rs/rusty_esp_dsp) [](https://github.com/Remade-With-Rust/rusty_esp_dsp/blob/main/LICENSE-MIT)
The pixel and sample kernels the **Janus** ESP32 family shares: colour
conversion, scaling, block difference, and the arithmetic that image, video and
audio would otherwise each write for themselves. Pure Rust, no C, no FFI,
`no_std` by default. Every kernel has a scalar twin kept in the tree as its
oracle, and every accelerated path is gated byte-identical against it.
* **One implementation, many callers.** Eight kernels — YUYV to 24-bit and
16-bit colour, 16-bit to 24-bit and back, half-size colour and grayscale,
YUYV to grayscale, and 16×16 block difference — used by the camera, the
encoder and the detector rather than duplicated in each.
* **Allocation-free by construction.** The chip reported its own free memory at
four stages of a run and the figure never moved: **not one of the eight
kernels allocates.** That is a property no code review can establish and a
board can.
* **Measured on the chip, because the laptop lies.** The optimisation target
for this package was originally chosen from a share table measured on a
development machine, where one kernel was 63.1% of the work. On the part the
code actually ships to, that kernel is **15.4%**, nothing exceeds 22.4%, and
four kernels sit in a band together. See below — it is the most useful thing
this package has produced.
## What has run on hardware
Measured on a Seeed XIAO ESP32-S3 Sense. The last column is the finding.
| kernel | share on the laptop | share on the chip | laptop faster by |
|---|---:|---:|---:|
| YUYV → 24-bit colour | 63.1% | **15.4%** | 85× |
| YUYV → 16-bit colour | 14.6% | **22.4%** | 534× |
| 16-bit → 24-bit colour | 6.3% | 19.4% | 1071× |
| half-size 16-bit colour | 5.8% | 17.7% | 1068× |
| 24-bit → 16-bit colour | 5.1% | 13.9% | 961× |
| YUYV → grayscale | 2.9% | 5.0% | 602× |
| half-size grayscale | 1.6% | 2.9% | 617× |
| block difference | 0.7% | 3.4% | 1617× |
**The last column explains the first two.** A laptop is between 85× and 1617×
faster at kernels doing comparable work per pixel. That nineteen-fold spread is
not a property of the chip — it is a map of which kernels the development
machine's compiler managed to vectorise, and the kernel where the gap is
smallest is the one it failed on. The host table had been ranking the host
compiler's blind spot.
A second hardware result, from adopting the house allocator: **four of eight
kernels moved by up to 8%, in both directions, with no allocator call inside
any of them.** It was where the buffers landed in memory, not the allocator's
speed — and a later release that deleted half the allocator's code left every
kernel within six parts per million. Placement, proven both ways.
Every number, with the run that produced it:
[`docs/LEDGER.md`](https://github.com/Remade-With-Rust/rusty_esp_dsp/blob/main/docs/LEDGER.md).
## Using it
```rust
use rusty_esp_dsp::ops;
// Sensor gives YUYV; the encoder wants RGB565. No allocation, no copy back.
ops::yuyv_to_rgb565(&yuyv, &mut rgb565, width, height)?;
// Cheap motion signal: sum of absolute differences over a 16x16 block.
let score = ops::sad_16x16(&previous, ¤t, stride);
```
## Two tracks
| track | what it is | this crate |
|---|---|---|
| **A** | `std` on ESP-IDF | `--features std` |
| **B** | `no_std` on `esp-hal` | default |
Hand-written vector kernels for the chip's own extension are **not** here yet:
the extension has no Rust intrinsics, so that is a spike before it is a
feature, and the share table above is what will aim it.
```sh
cargo test --workspace
cargo check -p rusty_esp_dsp --no-default-features --target riscv32imac-unknown-none-elf
```
## Part of Janus
**Janus** rebuilds the Espressif ESP32 and Arduino application portfolio as
independent, memory-safe Rust packages — so a hardware maker can ship a device
that the [MATA](https://www.mata.network) home computer discovers, catalogs honestly, adopts
under its own identity, and pays for. Ten packages, three layers, and the
dependency direction never reverses.
| layer | packages |
|---|---|
| **0 — the vocabulary** | [`rusty_esp_core`](https://crates.io/crates/rusty_esp_core) · [`rusty_esp_dsp`](https://crates.io/crates/rusty_esp_dsp) |
| **1 — the functions** | [`rusty_esp_image`](https://crates.io/crates/rusty_esp_image) · [`rusty_esp_video`](https://crates.io/crates/rusty_esp_video) · [`rusty_esp_audio`](https://crates.io/crates/rusty_esp_audio) · [`rusty_esp_signal`](https://crates.io/crates/rusty_esp_signal) · [`rusty_esp_mid`](https://crates.io/crates/rusty_esp_mid) · [`rusty_esp_iroh`](https://crates.io/crates/rusty_esp_iroh) |
| **2 — the surfaces** | [`rusty_esp_arduino`](https://crates.io/crates/rusty_esp_arduino) — the sketch facade · `espino` — the maker's CLI (not published) |
Every package is host-verified against an external oracle and keeps a ledger
in which no number appears without the run that produced it. **Five of seven
device profiles have now run their kill tests on real silicon**, three of them
over a Wi-Fi network the board hosts itself.
Also check out the rest of [Remade With Rust](https://github.com/remade-with-rust) — including
[`rusty_alloc`](https://crates.io/crates/rusty_alloc), the pure-Rust rebuild of
mimalloc that these firmwares run on, and
[`rusty_jpeg`](https://crates.io/crates/rusty_jpeg), the JPEG engine behind the
camera path — and our sister project
[remade_ffmpeg_rs](https://github.com/Remade-With-Rust/remade_ffmpeg_rs), a ground-up Rust rebuild of FFmpeg.
## About Mata Network
[Mata Network](https://www.mata.network) builds sovereign, self-hostable infrastructure.
**Remade With Rust** is our open-source home for the permissively-licensed
building blocks that work depends on.
## License
MIT OR Apache-2.0, at your option. See [LICENSE-MIT](https://github.com/Remade-With-Rust/rusty_esp_dsp/blob/main/LICENSE-MIT)
and [LICENSE-APACHE](https://github.com/Remade-With-Rust/rusty_esp_dsp/blob/main/LICENSE-APACHE).