Part of Ferramenta, a family of Rust tools.

Ferroni — Oniguruma-compatible regex engine

Oniguruma is the regex engine behind Ruby, PHP (mbstring), TextMate grammars, and tools like jq. It supports features that most regex libraries don't: named captures with multiple syntaxes, look-behind of variable length, conditional patterns, absent expressions, 886 Unicode properties, subexpression calls, and 12 syntax modes from Perl to POSIX.

Ferroni started with a practical goal: build a fast Rust core for syntax-highlighting and other scanner-heavy workloads without falling back to C. That requirement led straight to Oniguruma compatibility, because the surrounding ecosystems depend on features most regex engines skip.

So Ferroni does not wrap Oniguruma. It ports the engine into Rust, keeps the same structure and optimization pipeline, and then tunes the runtime path hard with tools like memchr. In the current reference suite, Ferroni is ahead of Oniguruma across the measured runtime cases while staying in the same peak-memory class on the large TypeScript scanner workload.

For syntax highlighting, Ferroni also includes a multi-pattern Scanner API compatible with vscode-oniguruma, used by Shiki, VS Code, and other TextMate-based highlighters.

Why Ferroni?

Built for runtime performance. Ferroni was driven by the need for a fast Rust core for syntax highlighting, not by a generic "rewrite C in Rust" exercise. In the current battle_bench reference suite it is ahead of Oniguruma across all measured runtime cases: scanner first-match, full-line tokenization, practical text scanning, and representative feature-heavy matching. On the measured large TypeScript scanner workload, peak RSS stays in the same ~15 MB class as Oniguruma.

Full Oniguruma compatibility. Named captures, variable-length look-behind, conditionals, absent expressions, Unicode properties, subexpression calls — everything the C engine supports, without linking against C. If your pattern works in Oniguruma, it works in Ferroni. Every opcode and optimization pass is ported 1:1 and verified by the ported upstream test suite -- including every upstream UTF-8 test from both Oniguruma and vscode-oniguruma.

Rust improves the operational story. Pure cargo build. Cross-compiles to wasm32-unknown-unknown. Easier to package in Rust-native stacks and downstream bindings, including N-API modules, without node-gyp or a local C compiler. Only the optional ffi feature — the C-vs-Rust benchmark harness, not needed to use the library — requires a local Oniguruma source snapshot (./scripts/prepare-oniguruma-sources.sh or FERRONI_ONIGURUMA_DIR). Rust also removes whole classes of C memory bugs structurally; C Oniguruma has a long history of memory-safety CVEs, while Ferroni keeps unsafe at 0.4%, all documented in ADR-002.

Built-in multi-pattern scanner. For syntax highlighting with TextMate grammars, Ferroni includes a vscode-oniguruma-compatible Scanner API — regex engine and scanner in a single dependency. cargo add ferroni and you're done.

Quick start

Add to your Cargo.toml:

[dependencies]
ferroni = "1"

MSRV: Rust 1.94 (enforced in CI).

Regex

use ferroni::prelude::*;

fn main() -> Result<(), RegexError> {
    let re = Regex::new(r"(?<year>\d{4})-(?<month>\d{2})-(?<day>\d{2})")?;

    let caps = re.captures("Date: 2026-02-12").unwrap();
    assert_eq!(caps.get(0).unwrap().as_str(), "2026-02-12");
    assert_eq!(caps.name("year").unwrap().as_str(), "2026");
    assert_eq!(caps.name("month").unwrap().as_str(), "02");
    Ok(())
}

Scanner API

The Scanner matches multiple patterns simultaneously -- the core operation behind TextMate-based syntax highlighting. Results include UTF-16 position mapping for direct use with vscode-textmate and Shiki.

use ferroni::scanner::{Scanner, ScannerFindOptions};

let mut scanner = Scanner::new(&[
    r"\b(function|const|let|var)\b",  // keywords
    r#""[^"]*""#,                      // strings
    r"//.*$",                          // comments
]).unwrap();

let code = r#"const x = "hello" // greeting"#;
let m = scanner.find_next_match(code, 0, ScannerFindOptions::NONE).unwrap();

assert_eq!(m.index, 0); // pattern 0 matched first ("const")
assert_eq!(m.capture_indices[0].start, 0);
assert_eq!(m.capture_indices[0].end, 5);

For fine-grained control, use RegexBuilder:

use ferroni::prelude::*;

let re = Regex::builder(r"hello")
    .case_insensitive(true)
    .build()
    .unwrap();
assert!(re.is_match("Hello World"));

The full C-ported API is also available for advanced usage:

use ferroni::regcomp::onig_new;
use ferroni::regexec::onig_search;
use ferroni::oniguruma::*;
use ferroni::regsyntax::OnigSyntaxOniguruma;

let reg = onig_new(
    b"\\d{4}-\\d{2}-\\d{2}",
    ONIG_OPTION_NONE,
    &ferroni::encodings::utf8::ONIG_ENCODING_UTF8,
    &OnigSyntaxOniguruma,
).unwrap();

let input = b"Date: 2026-02-12";
let (result, region) = onig_search(
    &reg, input, input.len(), 0, input.len(),
    Some(OnigRegion::new()), ONIG_OPTION_NONE,
);

assert!(result >= 0);
assert_eq!(result, 6); // match starts at byte 6

Both APIs have a runnable example in examples/:

cargo run --example basic_match       # matching, captures, iteration
cargo run --example scanner_tokenize  # multi-pattern scanner, UTF-16 offsets

Supported features

Scanner -- multi-pattern matching with result caching, two search strategies (RegSet for short strings, per-regex for long strings), and automatic UTF-16 position mapping. API-compatible with vscode-oniguruma.

Full Oniguruma regex -- every feature from the C engine:

All Perl/Ruby/Python syntax -- (?:...), (?=...), (?!...), (?<=...), (?<!...), (?>...)
Named captures -- (?<name>...), (?'name'...), (?P<name>...)
Backreferences -- \k<name>, \g<name>, relative \g<-1>
Conditionals -- (?(cond)T|F)
Absent expressions -- (?~...)
Unicode properties -- \p{Script_Extensions=Greek}, \p{Lu}, \p{Emoji} (886 names)
Grapheme clusters -- \X, text segment boundaries \y, \Y
Callouts -- (?{...}), (*FAIL), (*MAX{n}), (*COUNT), (*CMP)
12 syntax modes -- Oniguruma, Ruby, Perl, Perl_NG, Python, Java, Emacs, Grep, GNU, POSIX Basic/Extended, ASIS
Safety limits -- retry, time, stack, subexp call depth (global + per-search)

Performance

This summary draws on a recorded battle_bench run. Scanner comparisons use complete, unmodified Shiki TextMate grammars for TypeScript, CSS, and Rust. It shows the direction across the measured workloads, not a promise for every pattern or machine. Raw results, inputs, and measurement context are in Benchmark Results.

Scanner workloads

For syntax highlighting, Ferroni consistently spends less time finding the next match and tokenizing lines than Oniguruma. The advantage holds across all three production grammars, from roughly an order of magnitude to much larger wins on TypeScript first-match latency. Compiling the full TypeScript grammar is also faster in Ferroni, while Oniguruma still starts faster on the smaller Rust and CSS grammars. In practice, Ferroni's strongest fit is a scanner that compiles a grammar once and uses it repeatedly.

Search and matching

On the measured log-style searches, Ferroni is faster than Oniguruma for both matches and misses, capture-based extraction, timestamps, and multi-pattern search. The advantage ranges from modest to several-fold, spread across cases rather than resting on one standout case. Ferroni also comes out ahead across the measured Oniguruma-compatible matching patterns, including lookaround, Unicode properties, backreferences, and alternation.

When patterns fit the narrower syntax of Rust's regex crate, that engine often has the throughput advantage. It is a good choice for plain-text matching when its feature set is enough; Ferroni is for workloads that need Oniguruma features and practical scanner speed.

Compilation and memory

Compile-time results are mixed. Ferroni compiles the full TypeScript grammar faster, but Oniguruma has an edge on Rust and CSS grammar startup and on several individual patterns. The regex crate can take longer to compile, especially for capture-heavy patterns. In a separate process-level memory measurement, Ferroni and Oniguruma peak in the same approximate range while compiling and scanning a large TypeScript workload. Details and methodology are in the Memory Measurements.

Ferroni keeps benchmark suites separated by purpose:

Internal suite (regression_bench, Rust-only, regression/optimization work):

cargo bench
cargo bench --bench regression_bench

# compare two local baselines
cargo bench --bench regression_bench -- --save-baseline main
# switch to the branch you want to compare
cargo bench --bench regression_bench -- --baseline main

Reference suite (battle_bench, Ferroni vs Oniguruma for publishable numbers):
```
# one-time setup
./scripts/prepare-oniguruma-sources.sh

cargo bench --features ffi --bench battle_bench
```
Exact external input pins for this suite live in benches/battle_inputs.toml.
Memory comparison (process-isolated peak RSS on the large TypeScript scanner workload):
```
./scripts/run-battle-memory.sh
```

HTML report:

open target/criterion/report/index.html

Architecture

Each C source file maps 1:1 to a Rust module (ADR-001):

C File	Rust Module	Purpose
regparse.c	`regparse.rs`	Pattern parser
regcomp.c	`regcomp.rs`	AST-to-bytecode compiler
regexec.c	`regexec.rs`	VM executor
regint.h	`regint.rs`	Internal types and opcodes
oniguruma.h	`oniguruma.rs`	Public types and constants
regenc.c	`regenc.rs`	Encoding trait
regsyntax.c	`regsyntax.rs`	12 syntax definitions
regset.c	`regset.rs`	Multi-regex search (RegSet)
regerror.c	`regerror.rs`	Error messages
regtrav.c	`regtrav.rs`	Capture tree traversal
unicode.c	`unicode/mod.rs`	Unicode tables and segmentation
--	`scanner.rs`	Multi-pattern scanner for syntax highlighting

Compilation pipeline (same as C):

onig_new() -> onig_compile()
  -> onig_parse_tree()     (pattern -> AST)
  -> reduce_string_list()  (merge adjacent strings)
  -> tune_tree()           (6 optimization sub-passes)
  -> compile_tree()        (AST -> VM bytecode)
  -> set_optimize_info()   (extract search strategy)

Scope

Ferroni targets ASCII/UTF-8 workloads. The following are intentionally not included:

27 of 29 encodings -- only ASCII and UTF-8 (ADR-003)
POSIX/GNU API -- regcomp/regexec/regfree (ADR-012)
C memory management -- replaced by Rust's Drop trait
onig_new_deluxe -- C-specific allocation, use onig_new() instead

Running tests

Debug builds need a larger thread stack; the required sizes are stated once in ADR-013.

# Full UTF-8 suite (requires increased stack for debug builds)

RUST_MIN_STACK=268435456 cargo test --test compat_utf8 -- --test-threads=1

# Other suites
cargo test --test compat_syntax
cargo test --test compat_options
cargo test --test compat_regset
RUST_MIN_STACK=268435456 cargo test --test compat_back -- --test-threads=1

Warning: Never run cargo test -- --ignored -- the conditional_recursion_complex test intentionally hangs.

Test parity

Every upstream test that targets UTF-8 is ported 1:1. EUC-JP tests are out of scope (ADR-003).

Upstream file	Encoding	Upstream	Ferroni	Status
test_utf8.c	UTF-8	1,554	1,561	100%
test_back.c	UTF-8	1,225	1,225	100%
test_syntax.c	UTF-8	144	144	100%
test_options.c	UTF-8	47	48	100%
test_regset.c	UTF-8	4	15	100%
testc.c	EUC-JP	785	—	out of scope
C total (UTF-8)		2,974	2,993	100%
vscode-oniguruma tests	UTF-16	15	25	100%

On top of the ported upstream cases, Ferroni adds Rust-native tests for API integration, edge cases, error paths, and coverage gaps. The tree holds 2,195 #[test] functions in total (some compat functions bundle multiple upstream cases, so this is higher than the parity totals above). The count is derived from the tree by scripts/count-tests.sh, which is the single source for the test counts quoted in this README and in CONTRIBUTING.

C Oniguruma has no coverage reporting. Ferroni's test suite is a strict superset of the upstream tests.

Line coverage: 87.7%, gated at 87%. Measured with cargo-llvm-cov by scripts/coverage.sh, which CI runs on every pull request and which fails the build below the gate -- the threshold is defined in that script. Generated Unicode tables, the FFI bindings and the test files themselves are excluded from the measurement. 42 deeply recursive tests are skipped under LLVM instrumentation (stack overflow from coverage bookkeeping) but pass in normal cargo test.

Architecture decision records

ADR	Decision
001	1:1 structural parity with C original
002	Unsafe code policy
003	Encoding scope: ASCII and UTF-8 only
004	C-to-Rust translation patterns
005	Idiomatic Rust API layer
006	Scanner API for TextMate tokenization
007	SIMD-accelerated search via memchr
008	Rust-only optimizations and performance philosophy
009	Dependency philosophy
010	Benchmark strategy
011	Test strategy and C test suite parity
012	POSIX and GNU API not ported
013	Stack overflow mitigation in debug builds
014	Porting bugs: lessons learned

Contributing

Contributions are welcome! Please read CONTRIBUTING.md and review the ADRs before submitting a PR.

Acknowledgments

Ferroni is built on the work of K. Kosako and the Oniguruma contributors. The C original powers regex in Ruby, PHP, TextMate, jq, and many other projects. The Scanner API and its test suite are based on vscode-oniguruma by Microsoft and the VS Code team.

License

BSD-2-Clause (same as Oniguruma)

Ferroni is the one Ferramenta tool that is not published under MIT OR Apache-2.0: it is a line-by-line port of Oniguruma's C source, so it stays under the BSD-2-Clause license of the code it is derived from.

More from Ferramenta

Ferramenta — A family of Rust tools.

The content pipeline

Tool	Job
ferriki	Shiki-compatible syntax highlighting
ferromark	Markdown to HTML with a secure default and every GFM extension included.

The language workshop

Tool	Job
ferrolex	Spell checking for text and code
ferrocat	Translation catalog engine
palamedes	Internationalization for TypeScript applications

On the workbench

Tool	Job
ferrovia	SVGO-compatible SVG optimizer
ferralk	Glob matching and parallel filesystem walking
ferrugo	PDF previews for untrusted files

ferroni 1.4.2