ferroni 2.0.0

Oniguruma's regex engine in pure Rust: full Oniguruma syntax, a vscode-oniguruma-compatible TextMate scanner, no C toolchain
Documentation
<!-- This file is generated by mdtheme. Edit README.md.src instead. -->

Part of [Ferramenta](https://ferramenta.dev), a family of Rust tools.

# Ferroni — Oniguruma's regex engine, modernized in Rust

[![Powered by Sebastian Software](https://img.shields.io/badge/Powered_by-Sebastian_Software-005164?style=flat)](https://oss.sebastian-software.com) [![crates.io](https://img.shields.io/crates/v/ferroni?style=flat-square&logo=rust&label=crates.io)](https://crates.io/crates/ferroni)
[![docs.rs](https://img.shields.io/docsrs/ferroni?style=flat-square&logo=docsdotrs&label=docs.rs)](https://docs.rs/ferroni)
[![CI](https://img.shields.io/github/actions/workflow/status/sebastian-software/ferroni/ci.yml?branch=main&style=flat-square&logo=github&label=CI)](https://github.com/sebastian-software/ferroni/actions)
[![Coverage gate at least 87 percent](https://img.shields.io/badge/coverage%20gate-%E2%89%A5%2087%25-blue?style=flat-square)](https://github.com/sebastian-software/ferroni/blob/main/.github/workflows/ci.yml)
[![MSRV 1.94](https://img.shields.io/badge/MSRV-1.94-blue?style=flat-square&logo=rust)](CONTRIBUTING.md#getting-started)
[![License](https://img.shields.io/badge/license-BSD--2--Clause-blue?style=flat-square)](https://github.com/sebastian-software/ferroni/blob/main/LICENSE)

<p align="center">
  Evidence, not badges:
  <a href="https://ferroni.dev/guide/compatibility#test-parity">test parity with the C suite</a> &middot;
  <a href="https://ferroni.dev/perf/engine-comparison">benchmarks with raw data</a> &middot;
  <a href="https://ferroni.dev/adr/002-unsafe-code-policy">unsafe code policy</a>
</p>

---

**Full Oniguruma syntax for Rust: look-behind, backreferences, conditionals,
Unicode properties, and the multi-pattern scanner that TextMate grammars run
on. One `cargo add`, no C compiler, memory-safe, and faster than the C
original.**

[Oniguruma](https://github.com/kkos/oniguruma) is the regex engine TextMate
grammars are written for, which puts it underneath VS Code, Shiki and most
syntax highlighters. jq and PHP's mbregex use it too. Its C project
[ended in April 2025](https://github.com/kkos/oniguruma#readme). Ferroni ports
the engine to Rust line by line, checks it against the upstream test suite,
and adds what C Oniguruma never shipped: Unicode 18.0 data, a
vscode-oniguruma-compatible scanner, a compile-time check for catastrophic
backtracking, and an idiomatic Rust API.

- **Same engine, verified.** All 2,974 upstream UTF-8 test cases pass. The port
  keeps Oniguruma's parser, compiler and optimizer, so a pattern behaves as it
  does in C ([how compatibility is checked]https://ferroni.dev/guide/compatibility).
- **Easy to add.** `cargo add ferroni` is the whole setup: no C toolchain, no
  bindgen, six common dependencies. An idiomatic API of `Regex`, `Captures`
  and match iterators sits on top, and a compiled `Regex` is `Send + Sync`
  ([getting started]https://ferroni.dev/guide/getting-started).
- **Safe with untrusted input.** Memory-safe Rust with `unsafe` confined to
  the links between parse-tree nodes, timeouts and backtracking limits per
  search, and fuzzing on every pull request
  ([untrusted input]https://ferroni.dev/guide/untrusted-input).
- **Faster than the original.** Ahead of C Oniguruma in every measured
  workload, from everyday text processing to syntax highlighting
  ([performance]#performance).

## Quick start

```bash
cargo add ferroni
```

```rust
use ferroni::prelude::*;

fn main() -> Result<(), RegexError> {
    // A look-behind and named groups: Oniguruma syntax, a familiar API.
    let re = Regex::new(r"(?<=Date: )(?<year>\d{4})-(?<month>\d{2})-(?<day>\d{2})")?;

    let caps = re.captures("Date: 2026-02-12").unwrap();
    assert_eq!(caps.get(0).unwrap().as_str(), "2026-02-12");
    assert_eq!(caps.name("year").unwrap().as_str(), "2026");
    Ok(())
}
```

Syntax highlighters ask many patterns at once which one matches next.
`Scanner` answers that with the shape of vscode-oniguruma's scanner, UTF-16
offsets included:

```rust
use ferroni::prelude::*;

let mut scanner = Scanner::new(&[
    r"\b(function|const|let|var)\b", // 0: keywords
    r#""[^"]*""#,                     // 1: strings
    r"//.*$",                         // 2: comments
]).unwrap();

let line = r#"const x = "hello" // greeting"#;
let m = scanner.find_next_match(line, 0, ScannerFindOptions::NONE).unwrap();

assert_eq!(m.index, 0); // the keyword pattern matched first
assert_eq!(m.captures()[0].end, 5);
```

The guide continues with
[captures and iteration](https://ferroni.dev/guide/getting-started#captures),
[options and syntax modes](https://ferroni.dev/guide/getting-started#compile-time-options),
[the Scanner API](https://ferroni.dev/guide/getting-started#the-scanner-api),
[error handling](https://ferroni.dev/guide/getting-started#handling-errors) and
[the low-level C API](https://ferroni.dev/guide/getting-started#the-low-level-c-api).
Runnable versions of both snippets are in [`examples/`](examples):
`cargo run --example basic_match` and `cargo run --example scanner_tokenize`.

## When to use Ferroni

Ferroni is for code that needs Oniguruma's syntax or behavior:

- **Syntax highlighting and editors.** TextMate grammars rely on look-behind,
  `\G` anchors, backreferences and Oniguruma's own quantifier syntax.
  [Ferriki]https://ferriki.dev, a Shiki-compatible highlighter, tokenizes
  them with Ferroni's scanner.
- **Patterns the `regex` crate cannot run.** Look-around, backreferences,
  conditionals, subexpression calls and recursion, absent expressions,
  possessive quantifiers, and twelve syntax modes from Perl and Ruby to POSIX.
- **Dropping the C library.** If you link C Oniguruma today, Ferroni runs the
  same UTF-8 patterns without a C toolchain in your build.

When your patterns fit the syntax of Rust's
[`regex`](https://crates.io/crates/regex) crate, use that instead: it
guarantees linear-time matching and is usually faster on plain text search.

## Features

- **Full Oniguruma syntax:** look-ahead and variable-length look-behind, atomic
  groups, possessive quantifiers, named groups in three spellings,
  backreferences by number, name and relative position, conditionals
  `(?(cond)T|F)`, absent expressions `(?~...)`, subexpression calls
  `\g<name>`, and callouts.
- **Unicode 18.0:** 914 property names such as `\p{Greek}`, `\p{Lu}` and
  `\p{Emoji}`, grapheme clusters `\X`, text segment boundaries `\y` and `\Y`,
  and full case folding, so `(?i)straße` finds `STRASSE`. C Oniguruma's last
  release ships Unicode 16.0
  ([ADR-015]https://ferroni.dev/adr/015-unicode-data-from-ucd).
- **12 syntax modes:** Oniguruma, Ruby, Perl, Perl_NG, Python, Java, Emacs,
  Grep, GNU, POSIX Basic, POSIX Extended and ASIS.
- **A scanner for highlighters:** vscode-oniguruma's multi-pattern API with
  UTF-16 offsets, a result cache per string, and a pattern cache that scanners
  share ([ADR-006]https://ferroni.dev/adr/006-scanner-api).
- **Limits:** time, retry, stack and nesting limits, process-wide or per
  search.
- **Two APIs:** the idiomatic `Regex` API, and every C entry point under its
  original name (`onig_new`, `onig_search`, ...).

## Performance

How Ferroni compares with seven other engines on three kinds of work. Each
figure is the geometric mean over every task in the workload, on two test
machines.

<!-- engine-comparison -->
| Ferroni compared with | Everyday patterns | Advanced patterns | Syntax highlighting |
| --- | ---: | ---: | ---: |
| Oniguruma (C) | 2.2× faster | 1.4× faster | 12× faster |
| Shiki JS | – | – | 6.1× faster |
| Onigmo | 2.6× faster | 1.5× faster\* | 34× faster |
| PCRE2 | 1.7× faster\* | 1.4× faster\* | 28× faster |
| PCRE2 JIT | 2.1× slower\* | 3.5× slower\* | 9.7× faster |
| fancy-regex | 1.0× slower\* | 3.0× faster\* | 1.5× faster |
| regex | 1.5× slower\* | – | – |

- **Everyday patterns** (32 tasks): search and extraction in HTML, logs, chat with emoji, Markdown, JSON, CSV and source code, in syntax the regex crate also runs.
- **Advanced patterns** (17 tasks): look-around, backreferences, possessive groups, subexpression calls, absent expressions and grapheme clusters.
- **Syntax highlighting** (3 documents): the scanner calls Shiki makes for whole C, Java and PHP documents, in grammars every engine runs.

\* Ran only some of the tasks: the engine rejects a pattern or finds different matches. The figure covers the tasks it ran.

– Not measured: the engine lacks the syntax, or the multi-pattern API the workload needs.

fancy-regex is quoted in its fastest configuration per column: seek mode for everyday patterns and advanced patterns; RegexSet for syntax highlighting.
<!-- /engine-comparison -->

Measured 2026-10-09 on Blacksmith's macOS arm64 (Apple M4 Pro) and Linux
x86-64 (AMD EPYC) runners. Every engine first has to reproduce Oniguruma's
results before it is timed. These are observations on the measured workloads,
not a promise for every pattern or machine.

Where Ferroni is behind, the table says so: Rust's `regex` crate and PCRE2's
JIT are faster on most text tasks whose syntax they support, C Oniguruma wins
a few individual tasks, and grammar compilation is mixed. Per-host figures,
every task and the raw data:

- [Text processing, task by task]https://ferroni.dev/perf/engine-comparison#text-processing
- [Highlighting replays and complete TextMate grammars]https://ferroni.dev/perf/engine-comparison#highlighting
- [Compilation and short searches]https://ferroni.dev/perf/engine-comparison#short-searches-and-compilation
- [Tasks an engine cannot run]https://ferroni.dev/perf/engine-comparison#cases-an-engine-cannot-run
- [Memory use]https://ferroni.dev/perf/memory-measurements

## Correctness and safety

- **Test parity.** Every upstream test file for UTF-8 is ported case by case:
  2,974 C test cases and the vscode-oniguruma scanner tests, all passing
  ([parity table]https://ferroni.dev/guide/compatibility#test-parity). With
  Ferroni's own tests the tree holds **at least 2,500 `#[test]` functions**,
  counted by [`scripts/count-tests.sh`]scripts/count-tests.sh.
- **Coverage gate.** CI measures line coverage on every pull request and fails
  below the threshold set in [`scripts/coverage.sh`]scripts/coverage.sh.
- **Fuzzing.** Pattern compilation, matching and the scanner are fuzzed on
  every pull request, and for longer every week.
- **Memory safety.** C Oniguruma has a history of memory-safety CVEs in its
  buffer and string handling. Ferroni keeps that code in safe Rust; the
  remaining `unsafe`, 10 sites, covers only the links between parse-tree
  nodes during compilation ([ADR-002]https://ferroni.dev/adr/002-unsafe-code-policy).
- **Hostile patterns and input.** Per-search timeouts and retry limits stop
  runaway backtracking, and a compile-time check flags patterns such as
  `(a+)+` ([untrusted input]https://ferroni.dev/guide/untrusted-input).

## Scope

Ferroni supports ASCII and UTF-8, two of Oniguruma's 29 encodings. That covers
TextMate grammars, since vscode-oniguruma compiles every pattern as UTF-8, and
jq; PHP's mbregex in a non-UTF-8 encoding is not covered. The POSIX and GNU
APIs (`regcomp`, `regexec`) are not ported either
([what is covered](https://ferroni.dev/guide/compatibility#what-ferroni-covers)).

## Documentation

| Where | What |
| --- | --- |
| [Guide]https://ferroni.dev/guide/getting-started | Install, match, capture, scan, and handle errors |
| [Untrusted input]https://ferroni.dev/guide/untrusted-input | Limits, backtracking checks, and memory safety |
| [Compatibility]https://ferroni.dev/guide/compatibility | What is covered, test parity, and how the port maps to C |
| [Migrating to 2.0]https://ferroni.dev/guide/migrating-to-2 | The breaking changes of 2.0, with before and after code |
| [Performance]https://ferroni.dev/perf/engine-comparison | Every benchmark, with hosts, versions and raw data |
| [Architecture decisions]https://ferroni.dev/adr/001-one-to-one-parity-with-c-original | Why the port is built the way it is |
| [API reference]https://docs.rs/ferroni | Every type and function on docs.rs |

## Project status

Ferroni is feature-complete for its scope and maintained, with no feature work
planned. The scope is Oniguruma's syntax for ASCII and UTF-8, and the
vscode-oniguruma scanner.

The project maintains:

- bug fixes and parity fixes against C Oniguruma,
- the annual Unicode update,
- Rust toolchain and dependency upkeep,
- security fixes for the latest release. [SECURITY.md]SECURITY.md explains
  how to report a vulnerability and what to expect.

Not planned:

- the other 27 of Oniguruma's 29 encodings ([ADR-003]https://ferroni.dev/adr/003-encoding-scope-ascii-and-utf8-only),
- the POSIX and GNU APIs ([ADR-012]https://ferroni.dev/adr/012-posix-and-gnu-api-not-ported),
- an opt-in Onigmo syntax ([#159]https://github.com/sebastian-software/ferroni/issues/159).

Issues are welcome and are triaged as time allows. Pull requests are reviewed
against the [architecture decisions](https://ferroni.dev/adr/001-one-to-one-parity-with-c-original),
starting with structural parity with C Oniguruma. The `help wanted` label marks
the issues that are waiting for a contributor.

One integration is still open: Ferroni as a regex backend for
[syntect](https://github.com/trishume/syntect)
([#252](https://github.com/sebastian-software/ferroni/issues/252)).

## Contributing

Contributions are welcome. [CONTRIBUTING.md](CONTRIBUTING.md) covers the
build, the test suites, the benchmarks and the coverage gate. The
[architecture decisions](https://ferroni.dev/adr/001-one-to-one-parity-with-c-original)
describe the constraints a change has to keep, starting with structural
parity with the C original.

## Acknowledgments

Ferroni is built on the work of [K. Kosako](https://github.com/kkos) and
the Oniguruma contributors. The C original powers regex in
[PHP](https://www.php.net/), [TextMate](https://macromates.com/),
[jq](https://jqlang.github.io/jq/) and many other projects, and Ruby's
engine, [Onigmo](https://github.com/k-takata/Onigmo), started as a fork of it.
The Scanner API and its test suite are based on
[vscode-oniguruma](https://github.com/microsoft/vscode-oniguruma)
by Microsoft and the VS Code team.

Oniguruma ended in April 2025. Ferroni is an independent continuation by
Sebastian Software; it is not affiliated with or endorsed by the original
author.

## License

[BSD-2-Clause](LICENSE) (same as Oniguruma)

Ferroni is the one Ferramenta tool that is not published under `MIT OR
Apache-2.0`: it is a line-by-line port of Oniguruma's C source, so it stays
under the BSD-2-Clause license of the code it is derived from.

### <a href="https://ferramenta.dev"><img src="https://raw.githubusercontent.com/sebastian-software/ferramenta/main/app/assets/brand/logo-light.svg" width="24" height="24" alt="" /> More from Ferramenta</a>

[Ferramenta](https://ferramenta.dev) — A family of Rust tools.

**Engines**

| Tool | Job |
| --- | --- |
| [ferriki]https://ferriki.dev | Shiki-compatible syntax highlighting |
| [ferromark]https://ferromark.dev | Markdown to HTML, sanitized by default |
| [ferrolex]https://github.com/sebastian-software/ferrolex | Spell checking for text and code |
| [ferrocat]https://ferrocat.dev | Translation catalog engine |
| [ferralk]https://github.com/sebastian-software/ferralk | Glob matching and parallel filesystem walking |
| [ferrugo]https://github.com/sebastian-software/ferrugo | PDF previews for untrusted files |

**Applications**

| Tool | Job |
| --- | --- |
| [palamedes]https://palamedes.dev | Internationalization for TypeScript applications |
| [dalo]https://dalo.sh | Your team's agent setup, versioned like code |

---

<p align="center">
  <a href="https://oss.sebastian-software.com"><img src="https://raw.githubusercontent.com/sebastian-software/oss.sebastian-software.com/main/app/assets/logo-software.svg" alt="Sebastian Software" width="160" /></a><br />
  TypeScript, React &amp; Rust consulting<br />
  Experts in Agentic Software Development<br />
  <a href="https://sebastian-software.de">Work with us</a> · <a href="https://oss.sebastian-software.com">More open source</a>
</p>

<p align="center">Copyright &copy; 2026 Sebastian Software GmbH</p>