ferroni 1.1.0

Syntax highlighting in VS Code, Shiki, and every editor built on TextMate grammars runs on two things: an Oniguruma regex engine and a multi-pattern scanner. Today, that means C code with native bindings via vscode-oniguruma.

Ferroni puts both into a single Rust crate. Same regex semantics, same Scanner API, no C compiler needed. Just cargo build.

It is a line-by-line port of Oniguruma's C source -- same structure, same opcodes, same optimization passes -- with SIMD-vectorized search via memchr layered on top. The result: up to 6x faster than C on full-text scanning, while an idiomatic Rust API (Regex::new(), typed errors, Match/Captures) keeps the ergonomics clean.

Why Ferroni?

Regex engine + scanner in one crate. If you're building a syntax highlighter, a TextMate grammar host, or anything that matches multiple patterns against source code, you used to need C Oniguruma plus native bindings. Ferroni gives you both the regex engine and the vscode-oniguruma-compatible Scanner API in a single dependency. cargo add ferroni and you're done.

No more CVEs from C. C Oniguruma has a track record of memory safety vulnerabilities -- CVE-2019-13224 (CVSS 9.8), CVE-2019-19204, CVE-2019-19246, CVE-2019-19012, CVE-2019-13225 -- affecting Ruby, PHP, and anything linking against it. Ferroni eliminates buffer overflows, use-after-free, and NULL dereferences structurally through Rust's type system. 0.4% unsafe code, all documented in ADR-005.

Drop-in compatible. If your pattern works in Oniguruma, it works in Ferroni. Every opcode, every optimization pass is ported 1:1 from C and verified by 1,882 tests from three independent sources.

No C toolchain required. Pure cargo build. Cross-compiles to wasm32-unknown-unknown. Ship it as a Node.js native module via napi-rs without node-gyp or a C compiler on the user's machine.

Quick start

Add to your Cargo.toml:

[dependencies]
ferroni = "1"

Regex

use ferroni::prelude::*;

fn main() -> Result<(), RegexError> {
    let re = Regex::new(r"(?<year>\d{4})-(?<month>\d{2})-(?<day>\d{2})")?;

    let caps = re.captures("Date: 2026-02-12").unwrap();
    assert_eq!(caps.get(0).unwrap().as_str(), "2026-02-12");
    assert_eq!(caps.name("year").unwrap().as_str(), "2026");
    assert_eq!(caps.name("month").unwrap().as_str(), "02");
    Ok(())
}

Scanner API

The Scanner matches multiple patterns simultaneously -- the core operation behind TextMate-based syntax highlighting. Results include UTF-16 position mapping for direct use with vscode-textmate and Shiki.

use ferroni::scanner::{Scanner, ScannerFindOptions};

let mut scanner = Scanner::new(&[
    r"\b(function|const|let|var)\b",  // keywords
    r#""[^"]*""#,                      // strings
    r"//.*$",                          // comments
]).unwrap();

let code = r#"const x = "hello" // greeting"#;
let m = scanner.find_next_match(code, 0, ScannerFindOptions::NONE).unwrap();

assert_eq!(m.index, 0); // pattern 0 matched first ("const")
assert_eq!(m.capture_indices[0].start, 0);
assert_eq!(m.capture_indices[0].end, 5);

For fine-grained control, use RegexBuilder:

use ferroni::prelude::*;

let re = Regex::builder(r"hello")
    .case_insensitive(true)
    .build()
    .unwrap();
assert!(re.is_match("Hello World"));

The full C-ported API is also available for advanced usage:

use ferroni::regcomp::onig_new;
use ferroni::regexec::onig_search;
use ferroni::oniguruma::*;
use ferroni::regsyntax::OnigSyntaxOniguruma;

let reg = onig_new(
    b"\\d{4}-\\d{2}-\\d{2}",
    ONIG_OPTION_NONE,
    &ferroni::encodings::utf8::ONIG_ENCODING_UTF8,
    &OnigSyntaxOniguruma,
).unwrap();

let input = b"Date: 2026-02-12";
let (result, region) = onig_search(
    &reg, input, input.len(), 0, input.len(),
    Some(OnigRegion::new()), ONIG_OPTION_NONE,
);

assert!(result >= 0);
assert_eq!(result, 6); // match starts at byte 6

Supported features

Scanner -- multi-pattern matching with result caching, two search strategies (RegSet for short strings, per-regex for long strings), and automatic UTF-16 position mapping. API-compatible with vscode-oniguruma.

Full Oniguruma regex -- every feature from the C engine:

• All Perl/Ruby/Python syntax -- (?:...), (?=...), (?!...), (?<=...), (?<!...), (?>...)
• Named captures -- (?<name>...), (?'name'...), (?P<name>...)
• Backreferences -- \k<name>, \g<name>, relative \g<-1>
• Conditionals -- (?(cond)T|F)
• Absent expressions -- (?~...)
• Unicode properties -- \p{Script_Extensions=Greek}, \p{Lu}, \p{Emoji} (886 names)
• Grapheme clusters -- \X, text segment boundaries \y, \Y
• Callouts -- (?{...}), (*FAIL), (*MAX{n}), (*COUNT), (*CMP)
• 12 syntax modes -- Oniguruma, Ruby, Perl, Perl_NG, Python, Java, Emacs, Grep, GNU, POSIX Basic/Extended, ASIS
• Safety limits -- retry, time, stack, subexp call depth (global + per-search)

Performance

Ferroni wins 31 of 42 execution benchmarks against C Oniguruma at -O3. Criterion, Apple M1 Ultra. Bold = faster engine.

Highlights

The largest gains come from SIMD-vectorized search via memchr -- NEON on ARM, SSE2/AVX2 on x86-64 -- replacing C's hand-written byte loops with vectorized scans. See ADR-006.

The Scanner warm path (all patterns served from cache, the steady-state in a syntax highlighter) runs at 24 ns -- within 6% of the C implementation. No heap allocation on cache hits.

Compilation is 1.2-1.7x slower than C (Rust allocates more per compilation), but patterns are compiled once and matched millions of times.

Regex execution

Regex compilation

Running benchmarks

cargo bench --features ffi               # full suite (~8 min)
cargo bench --features ffi -- compile    # specific group
cargo bench --features ffi -- scanner    # scanner API benchmarks
cargo bench --features ffi -- "large_"   # pattern filter
# HTML report: target/criterion/report/index.html

Architecture

Each C source file maps 1:1 to a Rust module (ADR-001):

Compilation pipeline (same as C):

onig_new() -> onig_compile()
  -> onig_parse_tree()     (pattern -> AST)
  -> reduce_string_list()  (merge adjacent strings)
  -> tune_tree()           (6 optimization sub-passes)
  -> compile_tree()        (AST -> VM bytecode)
  -> set_optimize_info()   (extract search strategy)

Scope

Ferroni targets ASCII/UTF-8 workloads. The following are intentionally not included:

• 27 of 29 encodings -- only ASCII and UTF-8 (ADR-002)
• POSIX/GNU API -- regcomp/regexec/regfree (ADR-007)
• C memory management -- replaced by Rust's Drop trait
• onig_new_deluxe -- C-specific allocation, use onig_new() instead

Running tests

# Full UTF-8 suite (requires increased stack for debug builds)
RUST_MIN_STACK=268435456 cargo test --test compat_utf8 -- --test-threads=1

# Other suites
cargo test --test compat_syntax
cargo test --test compat_options
cargo test --test compat_regset
RUST_MIN_STACK=268435456 cargo test --test compat_back -- --test-threads=1

Warning: Never run cargo test -- --ignored -- the conditional_recursion_complex test intentionally hangs.

Test coverage

1,882 tests from three independent sources:

• 1,554 ported 1:1 from C Oniguruma's test suite
• 25 from vscode-oniguruma's TypeScript tests (Scanner API, UTF-16 mapping)
• 303 Rust-specific tests for edge cases, error paths, and gaps in the upstream suites

C Oniguruma has no coverage reporting. Ferroni's test suite is a strict superset.

Coverage measured with cargo-llvm-cov, reported to Codecov.

Architecture decision records

Contributing

Contributions are welcome! Please read CONTRIBUTING.md and review the ADRs before submitting a PR.

Acknowledgments

Ferroni is built on the work of K. Kosako and the Oniguruma contributors. The C original powers regex in Ruby, PHP, TextMate, and many other projects. The Scanner API and its test suite are based on vscode-oniguruma by Nicolò Ribaudo and the VS Code team.

License

BSD-2-Clause (same as Oniguruma)

Copyright 2026 Sebastian Software GmbH