Part of [Ferramenta](https://ferramenta.dev), a family of Rust tools.
# Ferroni — Oniguruma-compatible regex engine
[](https://oss.sebastian-software.com) [](https://crates.io/crates/ferroni)
[](https://docs.rs/ferroni)
[](https://github.com/sebastian-software/ferroni/actions)
[](https://github.com/sebastian-software/ferroni/blob/main/.github/workflows/ci.yml)
[](CONTRIBUTING.md#getting-started)
[](https://github.com/sebastian-software/ferroni/blob/main/LICENSE)
<p align="center">
Evidence, not badges:
<a href="#test-parity">test parity with the C suite</a> ·
<a href="https://github.com/sebastian-software/ferroni/blob/main/.github/workflows/ci.yml">coverage gate enforced in CI</a> ·
<a href="https://sebastian-software.github.io/ferroni/adr/002-unsafe-code-policy">unsafe code policy</a>
</p>
---
[Oniguruma](https://github.com/kkos/oniguruma) is the regex engine behind
[Ruby](https://www.ruby-lang.org/), [PHP](https://www.php.net/) (mbstring),
[TextMate](https://macromates.com/) grammars, and tools like
[jq](https://jqlang.github.io/jq/). It supports features that most regex
libraries don't: named captures with multiple syntaxes, look-behind of
variable length, conditional patterns, absent expressions, 886 Unicode
properties, subexpression calls, and 12 syntax modes from Perl to POSIX.
Ferroni started with a practical goal: build a fast Rust core for
syntax-highlighting and other scanner-heavy workloads without falling back to
C. That requirement led straight to Oniguruma compatibility, because the
surrounding ecosystems depend on features most regex engines skip.
So Ferroni does not wrap Oniguruma. It ports the engine into Rust, keeps the
same structure and optimization pipeline, and then tunes the runtime path
hard with tools like [`memchr`](https://crates.io/crates/memchr). In the
current reference suite, Ferroni is ahead of Oniguruma across the measured
runtime cases while staying in the same peak-memory class on the large
TypeScript scanner workload.
For syntax highlighting, Ferroni also includes a multi-pattern
[Scanner API](#scanner-api) compatible with
[vscode-oniguruma](https://github.com/microsoft/vscode-oniguruma),
used by [Shiki](https://shiki.style/), VS Code, and other TextMate-based
highlighters.
## Why Ferroni?
**Built for runtime performance.** Ferroni was driven by the need for a fast
Rust core for syntax highlighting, not by a generic "rewrite C in Rust"
exercise. In the current `battle_bench` reference suite it is ahead of
Oniguruma across all measured runtime cases: scanner first-match, full-line
tokenization, practical text scanning, and representative feature-heavy
matching. On the measured large TypeScript scanner workload, peak RSS stays
in the same ~15 MB class as Oniguruma.
**Full Oniguruma compatibility.** Named captures, variable-length
look-behind, conditionals, absent expressions, Unicode properties,
subexpression calls — everything the C engine supports, without linking
against C. If your pattern works in Oniguruma, it works in Ferroni. Every
opcode and optimization pass is ported 1:1 and verified by the
[ported upstream test suite](#test-parity) -- including every upstream UTF-8
test from both Oniguruma and vscode-oniguruma.
**Rust improves the operational story.** Pure `cargo build`.
Cross-compiles to `wasm32-unknown-unknown`. Easier to package in Rust-native
stacks and downstream bindings, including N-API modules, without `node-gyp`
or a local C compiler.
Only the optional `ffi` feature — the C-vs-Rust benchmark harness, not
needed to use the library — requires a local Oniguruma source snapshot
(`./scripts/prepare-oniguruma-sources.sh` or `FERRONI_ONIGURUMA_DIR`). Rust also removes whole classes of C
memory bugs structurally; C Oniguruma has a long history of memory-safety
CVEs, while Ferroni keeps `unsafe` at 0.4%, all documented in
[ADR-002](https://sebastian-software.github.io/ferroni/adr/002-unsafe-code-policy).
**Built-in multi-pattern scanner.** For syntax highlighting with TextMate
grammars, Ferroni includes a
[vscode-oniguruma-compatible Scanner API](#scanner-api) — regex engine and
scanner in a single dependency. `cargo add ferroni` and you're done.
## Quick start
Add to your `Cargo.toml`:
```toml
[dependencies]
ferroni = "1"
```
MSRV: Rust 1.94 (enforced in CI).
### Regex
```rust
use ferroni::prelude::*;
fn main() -> Result<(), RegexError> {
let re = Regex::new(r"(?<year>\d{4})-(?<month>\d{2})-(?<day>\d{2})")?;
let caps = re.captures("Date: 2026-02-12").unwrap();
assert_eq!(caps.get(0).unwrap().as_str(), "2026-02-12");
assert_eq!(caps.name("year").unwrap().as_str(), "2026");
assert_eq!(caps.name("month").unwrap().as_str(), "02");
Ok(())
}
```
### Scanner API
The Scanner matches multiple patterns simultaneously -- the core operation
behind TextMate-based syntax highlighting. Results include UTF-16 position
mapping for direct use with vscode-textmate and Shiki.
```rust
use ferroni::scanner::{Scanner, ScannerFindOptions};
let mut scanner = Scanner::new(&[
r"\b(function|const|let|var)\b", // keywords
r#""[^"]*""#, // strings
r"//.*$", // comments
]).unwrap();
let code = r#"const x = "hello" // greeting"#;
let m = scanner.find_next_match(code, 0, ScannerFindOptions::NONE).unwrap();
assert_eq!(m.index, 0); // pattern 0 matched first ("const")
assert_eq!(m.capture_indices[0].start, 0);
assert_eq!(m.capture_indices[0].end, 5);
```
For fine-grained control, use `RegexBuilder`:
```rust
use ferroni::prelude::*;
let re = Regex::builder(r"hello")
.case_insensitive(true)
.build()
.unwrap();
assert!(re.is_match("Hello World"));
```
<details>
<summary><strong>Low-level C-style API</strong></summary>
The full C-ported API is also available for advanced usage:
```rust
use ferroni::regcomp::onig_new;
use ferroni::regexec::onig_search;
use ferroni::oniguruma::*;
use ferroni::regsyntax::OnigSyntaxOniguruma;
let reg = onig_new(
b"\\d{4}-\\d{2}-\\d{2}",
ONIG_OPTION_NONE,
&ferroni::encodings::utf8::ONIG_ENCODING_UTF8,
&OnigSyntaxOniguruma,
).unwrap();
let input = b"Date: 2026-02-12";
let (result, region) = onig_search(
®, input, input.len(), 0, input.len(),
Some(OnigRegion::new()), ONIG_OPTION_NONE,
);
assert!(result >= 0);
assert_eq!(result, 6); // match starts at byte 6
```
</details>
Both APIs have a runnable example in [`examples/`](examples):
```bash
cargo run --example basic_match # matching, captures, iteration
cargo run --example scanner_tokenize # multi-pattern scanner, UTF-16 offsets
```
## Supported features
**Scanner** -- multi-pattern matching with result caching, two search
strategies (RegSet for short strings, per-regex for long strings), and
automatic UTF-16 position mapping. API-compatible with
[vscode-oniguruma](https://github.com/microsoft/vscode-oniguruma).
**Full Oniguruma regex** -- every feature from the C engine:
- All Perl/Ruby/Python syntax -- `(?:...)`, `(?=...)`, `(?!...)`, `(?<=...)`, `(?<!...)`, `(?>...)`
- Named captures -- `(?<name>...)`, `(?'name'...)`, `(?P<name>...)`
- Backreferences -- `\k<name>`, `\g<name>`, relative `\g<-1>`
- Conditionals -- `(?(cond)T|F)`
- Absent expressions -- `(?~...)`
- Unicode properties -- `\p{Script_Extensions=Greek}`, `\p{Lu}`, `\p{Emoji}` (886 names)
- Grapheme clusters -- `\X`, text segment boundaries `\y`, `\Y`
- Callouts -- `(?{...})`, `(*FAIL)`, `(*MAX{n})`, `(*COUNT)`, `(*CMP)`
- 12 syntax modes -- Oniguruma, Ruby, Perl, Perl_NG, Python, Java, Emacs, Grep, GNU, POSIX Basic/Extended, ASIS
- Safety limits -- retry, time, stack, subexp call depth (global + per-search)
## Performance
This summary draws on a recorded
[`battle_bench`](https://github.com/sebastian-software/ferroni/blob/main/benches/battle_bench.rs)
run. Scanner comparisons use complete, unmodified Shiki TextMate grammars for
TypeScript, CSS, and Rust. It shows the direction across the measured
workloads, not a promise for every pattern or machine. Raw results, inputs,
and measurement context are in
[Benchmark Results](https://sebastian-software.github.io/ferroni/perf/benchmark-results).
### Scanner workloads
For syntax highlighting, Ferroni consistently spends less time finding the next
match and tokenizing lines than Oniguruma. The advantage holds across all three
production grammars, from roughly an order of magnitude to much larger wins on
TypeScript first-match latency. Compiling the full TypeScript grammar is also
faster in Ferroni, while Oniguruma still starts faster on the smaller Rust and
CSS grammars. In practice, Ferroni's strongest fit is a scanner that compiles a
grammar once and uses it repeatedly.
### Search and matching
On the measured log-style searches, Ferroni is faster than Oniguruma for both
matches and misses, capture-based extraction, timestamps, and multi-pattern
search. The advantage ranges from modest to several-fold, spread across cases
rather than resting on one standout case. Ferroni also comes out ahead across
the measured Oniguruma-compatible matching patterns, including lookaround,
Unicode properties, backreferences, and alternation.
When patterns fit the narrower syntax of Rust's
[`regex`](https://crates.io/crates/regex) crate, that engine often has the
throughput advantage. It is a good choice for plain-text matching when its
feature set is enough; Ferroni is for workloads that need Oniguruma features
and practical scanner speed.
### Compilation and memory
Compile-time results are mixed. Ferroni compiles the full TypeScript grammar
faster, but Oniguruma has an edge on Rust and CSS grammar startup and on several
individual patterns. The `regex` crate can take longer to compile, especially
for capture-heavy patterns. In a separate process-level memory measurement,
Ferroni and Oniguruma peak in the same approximate range while compiling and
scanning a large TypeScript workload. Details and methodology are in the
[Memory Measurements](https://sebastian-software.github.io/ferroni/perf/memory-measurements).
<details>
<summary><strong>Running benchmarks</strong></summary>
Ferroni keeps benchmark suites separated by purpose:
- **Internal suite (`regression_bench`, Rust-only, regression/optimization work):**
```bash
cargo bench
cargo bench --bench regression_bench
cargo bench --bench regression_bench -- --save-baseline main
cargo bench --bench regression_bench -- --baseline main
```
- **Reference suite (`battle_bench`, Ferroni vs Oniguruma for publishable numbers):**
```bash
./scripts/prepare-oniguruma-sources.sh
cargo bench --features ffi --bench battle_bench
```
Exact external input pins for this suite live in
[`benches/battle_inputs.toml`](benches/battle_inputs.toml).
- **Memory comparison (process-isolated peak RSS on the large TypeScript scanner workload):**
```bash
./scripts/run-battle-memory.sh
```
- **HTML report:**
```bash
open target/criterion/report/index.html
```
</details>
## Architecture
Each C source file maps 1:1 to a Rust module ([ADR-001](https://sebastian-software.github.io/ferroni/adr/001-one-to-one-parity-with-c-original)):
| regparse.c | `regparse.rs` | Pattern parser |
| regcomp.c | `regcomp.rs` | AST-to-bytecode compiler |
| regexec.c | `regexec.rs` | VM executor |
| regint.h | `regint.rs` | Internal types and opcodes |
| oniguruma.h | `oniguruma.rs` | Public types and constants |
| regenc.c | `regenc.rs` | Encoding trait |
| regsyntax.c | `regsyntax.rs` | 12 syntax definitions |
| regset.c | `regset.rs` | Multi-regex search (RegSet) |
| regerror.c | `regerror.rs` | Error messages |
| regtrav.c | `regtrav.rs` | Capture tree traversal |
| unicode.c | `unicode/mod.rs` | Unicode tables and segmentation |
| -- | `scanner.rs` | Multi-pattern scanner for syntax highlighting |
**Compilation pipeline** (same as C):
```
onig_new() -> onig_compile()
-> onig_parse_tree() (pattern -> AST)
-> reduce_string_list() (merge adjacent strings)
-> tune_tree() (6 optimization sub-passes)
-> compile_tree() (AST -> VM bytecode)
-> set_optimize_info() (extract search strategy)
```
## Scope
Ferroni targets ASCII/UTF-8 workloads. The following are intentionally not included:
- **27 of 29 encodings** -- only ASCII and UTF-8 ([ADR-003](https://sebastian-software.github.io/ferroni/adr/003-encoding-scope-ascii-and-utf8-only))
- **POSIX/GNU API** -- `regcomp`/`regexec`/`regfree` ([ADR-012](https://sebastian-software.github.io/ferroni/adr/012-posix-and-gnu-api-not-ported))
- **C memory management** -- replaced by Rust's `Drop` trait
- **`onig_new_deluxe`** -- C-specific allocation, use `onig_new()` instead
## Running tests
Debug builds need a larger thread stack; the required sizes are stated once in
[ADR-013](https://sebastian-software.github.io/ferroni/adr/013-stack-overflow-debug-builds).
```bash
# Full UTF-8 suite (requires increased stack for debug builds)
RUST_MIN_STACK=268435456 cargo test --test compat_utf8 -- --test-threads=1
# Other suites
cargo test --test compat_syntax
cargo test --test compat_options
cargo test --test compat_regset
RUST_MIN_STACK=268435456 cargo test --test compat_back -- --test-threads=1
```
> **Warning:** Never run `cargo test -- --ignored` -- the
> `conditional_recursion_complex` test intentionally hangs.
## Test parity
Every upstream test that targets UTF-8 is ported 1:1. EUC-JP tests are
out of scope ([ADR-003](https://sebastian-software.github.io/ferroni/adr/003-encoding-scope-ascii-and-utf8-only)).
| test_utf8.c | UTF-8 | 1,554 | 1,561 | 100% |
| test_back.c | UTF-8 | 1,225 | 1,225 | 100% |
| test_syntax.c | UTF-8 | 144 | 144 | 100% |
| test_options.c | UTF-8 | 47 | 48 | 100% |
| test_regset.c | UTF-8 | 4 | 15 | 100% |
| testc.c | EUC-JP | 785 | — | out of scope |
| **C total (UTF-8)** | | **2,974** | **2,993** | **100%** |
| [vscode-oniguruma](https://github.com/microsoft/vscode-oniguruma) tests | UTF-16 | 15 | 25 | 100% |
On top of the ported upstream cases, Ferroni adds Rust-native tests for
API integration, edge cases, error paths, and coverage gaps. The tree holds
**2,195 `#[test]` functions** in total (some compat functions bundle multiple
upstream cases, so this is higher than the parity totals above). The count is
derived from the tree by [`scripts/count-tests.sh`](scripts/count-tests.sh),
which is the single source for the test counts quoted in this README and in
CONTRIBUTING.
C Oniguruma has no coverage reporting. Ferroni's test suite is a strict
superset of the upstream tests.
**Line coverage: 87.7%, gated at 87%.** Measured with
[cargo-llvm-cov](https://github.com/taiki-e/cargo-llvm-cov) by
[`scripts/coverage.sh`](scripts/coverage.sh), which CI runs on every pull
request and which fails the build below the gate -- the threshold is defined in
that script. Generated Unicode tables, the FFI bindings and the test files
themselves are excluded from the measurement. 42 deeply recursive tests are
skipped under LLVM instrumentation (stack overflow from coverage bookkeeping)
but pass in normal `cargo test`.
## Architecture decision records
| [001](https://sebastian-software.github.io/ferroni/adr/001-one-to-one-parity-with-c-original) | 1:1 structural parity with C original |
| [002](https://sebastian-software.github.io/ferroni/adr/002-unsafe-code-policy) | Unsafe code policy |
| [003](https://sebastian-software.github.io/ferroni/adr/003-encoding-scope-ascii-and-utf8-only) | Encoding scope: ASCII and UTF-8 only |
| [004](https://sebastian-software.github.io/ferroni/adr/004-c-to-rust-translation-patterns) | C-to-Rust translation patterns |
| [005](https://sebastian-software.github.io/ferroni/adr/005-idiomatic-rust-api-layer) | Idiomatic Rust API layer |
| [006](https://sebastian-software.github.io/ferroni/adr/006-scanner-api) | Scanner API for TextMate tokenization |
| [007](https://sebastian-software.github.io/ferroni/adr/007-simd-accelerated-search) | SIMD-accelerated search via memchr |
| [008](https://sebastian-software.github.io/ferroni/adr/008-rust-only-optimizations) | Rust-only optimizations and performance philosophy |
| [009](https://sebastian-software.github.io/ferroni/adr/009-dependency-philosophy) | Dependency philosophy |
| [010](https://sebastian-software.github.io/ferroni/adr/010-benchmark-strategy) | Benchmark strategy |
| [011](https://sebastian-software.github.io/ferroni/adr/011-test-strategy-and-c-test-parity) | Test strategy and C test suite parity |
| [012](https://sebastian-software.github.io/ferroni/adr/012-posix-and-gnu-api-not-ported) | POSIX and GNU API not ported |
| [013](https://sebastian-software.github.io/ferroni/adr/013-stack-overflow-debug-builds) | Stack overflow mitigation in debug builds |
| [014](https://sebastian-software.github.io/ferroni/adr/014-porting-bugs-lessons-learned) | Porting bugs: lessons learned |
## Contributing
Contributions are welcome! Please read [CONTRIBUTING.md](CONTRIBUTING.md)
and review the ADRs before submitting a PR.
## Acknowledgments
Ferroni is built on the work of [K. Kosako](https://github.com/kkos) and
the Oniguruma contributors. The C original powers regex in
[Ruby](https://www.ruby-lang.org/), [PHP](https://www.php.net/),
[TextMate](https://macromates.com/), [jq](https://jqlang.github.io/jq/),
and many other projects. The Scanner API and its test suite are based on
[vscode-oniguruma](https://github.com/microsoft/vscode-oniguruma)
by Microsoft and the VS Code team.
## License
[BSD-2-Clause](LICENSE) (same as Oniguruma)
Ferroni is the one Ferramenta tool that is not published under `MIT OR
Apache-2.0`: it is a line-by-line port of Oniguruma's C source, so it stays
under the BSD-2-Clause license of the code it is derived from.
### <a href="https://ferramenta.dev"><img src="https://raw.githubusercontent.com/sebastian-software/ferramenta/main/app/assets/brand/logo-light.svg" width="24" height="24" alt="" /> More from Ferramenta</a>
[Ferramenta](https://ferramenta.dev) — A family of Rust tools.
**The content pipeline**
| [ferriki](https://github.com/sebastian-software/ferriki) | Shiki-compatible syntax highlighting |
| [ferromark](https://sebastian-software.github.io/ferromark/) | Markdown to HTML with a secure default and every GFM extension included. |
**The language workshop**
| [ferrolex](https://github.com/sebastian-software/ferrolex) | Spell checking for text and code |
| [ferrocat](https://ferrocat.dev) | Translation catalog engine |
| [palamedes](https://palamedes.dev) | Internationalization for TypeScript applications |
**On the workbench**
| [ferrovia](https://github.com/sebastian-software/ferrovia) | SVGO-compatible SVG optimizer |
| [ferralk](https://github.com/sebastian-software/ferralk) | Glob matching and parallel filesystem walking |
| [ferrugo](https://github.com/sebastian-software/ferrugo) | PDF previews for untrusted files |
---
<p align="center">
<a href="https://oss.sebastian-software.com"><img src="https://raw.githubusercontent.com/sebastian-software/oss.sebastian-software.com/main/app/assets/logo-software.svg" alt="Sebastian Software" width="160" /></a><br />
TypeScript, React & Rust consulting<br />
Experts in Agentic Software Development<br />
<a href="https://sebastian-software.de">Work with us</a> · <a href="https://oss.sebastian-software.com">More open source</a>
</p>
<p align="center">Copyright © 2026 Sebastian Software GmbH</p>