ferriki 0.4.0

Reusable native Rust syntax highlighter with TextMate grammars
Documentation
<!-- This file is generated by mdtheme. Edit README.md.src instead. -->

Part of [Ferramenta](https://ferramenta.dev), a family of Rust tools.

# Ferriki

[![Powered by Sebastian Software](https://img.shields.io/badge/Powered_by-Sebastian_Software-005164?style=flat)](https://oss.sebastian-software.com) [![CI](https://github.com/sebastian-software/ferriki/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/sebastian-software/ferriki/actions/workflows/ci.yml)
[![Coverage gate >= 89%](https://img.shields.io/badge/coverage%20gate-%3E%3D%2089%25-brightgreen.svg)](./.github/workflows/ci.yml)
[![License: MIT OR Apache-2.0](https://img.shields.io/badge/license-MIT%20OR%20Apache--2.0-blue.svg)](./LICENSE-MIT)
[![Node >= 22.13.0](https://img.shields.io/badge/node-%3E%3D22.13.0-brightgreen.svg)](https://nodejs.org)

Ferriki is Shiki-compatible syntax highlighting with a leaner Rust core and
Node bindings.

It keeps the API shape people already know from Shiki, but removes the old
JS/WASM multi-engine direction from the runtime. The goal is simple: keep the
developer experience familiar, move the heavy lifting into Rust, and end up
with a smaller, cleaner architecture that is easier to reason about and easier
to ship.

## Why Ferriki

- Shiki-compatible where it matters: existing highlighting-oriented Node APIs stay recognizable.
- Rust-first by design: grammar handling, theme application, state management, and rendering belong in the native core.
- Shiki-derived standard assets should ship with Ferriki, but load lazily instead of living as an always-on bundled runtime catalog.
- Leaner runtime shape: no product dependency on the historical JS/WASM engine stack.
- Verifiable compatibility: the Node layer is checked against a strict mirrored Shiki release-tag suite, while the tokenizer is checked against a pinned vscode-textmate oracle.

Ferriki is for teams that like the Shiki contract, but want a cleaner native
foundation under it.

## Product Scope

Ferriki is intentionally narrow right now.

| Area | Status | Notes |
| --- | --- | --- |
| Core highlighting runtime | Integrated | Native Rust core, Node bindings, Shiki-compatible highlighting API |
| Direct outputs like `codeToHtml`, `codeToTokens`, `codeToHast` | Integrated | Part of the main product surface |
| `transformers`, `colorized-brackets` | Not integrated | These may exist in the mirrored compatibility workspace, but they are not part of the Ferriki product boundary |
| `markdown-it`, `rehype`, `VitePress integrations` | Out of scope | These are adapters on top of `codeToHtml` / `codeToHast`, so Ferriki does not treat them as product features |
| Future native extension lanes | Possible later | If Ferriki takes on these areas, the preferred direction is Rust-native ownership, not a permanent JS wrapper stack |

The mirrored Shiki workspace under [`node/compat/upstream/shiki`](node/compat/upstream/shiki) exists to verify compatibility claims. It is not a statement that every mirrored package is a Ferriki feature.

## Products

This repository has a Rust library and a Node package:

- [`ferriki`]src (repository root package): reusable Rust highlighter and renderer
- [`node/ferriki`]node/ferriki: the Node-facing package surface

The private [`ferriki-core`](crates/ferriki-core) crate hosts the N-API binding
over the same Rust runtime.

## Repository Layout

- [`src`]src: the `ferriki` crate, native highlighter and asset catalogs
- [`crates/ferriki-core`]crates/ferriki-core: N-API host
- [`node/ferriki`]node/ferriki: Node package
- [`node/compat/harness`]node/compat/harness: Ferriki-specific compatibility glue
- [`node/compat/upstream/shiki`]node/compat/upstream/shiki: strict upstream Shiki mirror
- [`node/compat/upstream/vscode-textmate`]node/compat/upstream/vscode-textmate: strict tokenizer source and test oracle
- [`adr`]adr: architecture decision records ([index]adr/README.md)

Contributor workflow, test lanes, and the mirror rules are documented in
[`CONTRIBUTING.md`](CONTRIBUTING.md).

User-facing documentation lives in the [`docs/`](docs) directory:

- [Ferriki API reference]docs/ferriki-api.md
- [Rust API and Ferromark integration]docs/rust-api.md
- [Shiki migration guide]docs/migrations/shiki-to-ferriki.md
- [Compatibility and support policy]docs/compatibility.md
- [Troubleshooting]docs/troubleshooting.md
- [Ferromark + Ardo example]docs/examples/ferromark-ardo.mjs

The repository root is intentionally Rust-first. All Node, npm, and upstream
compatibility machinery lives under [`node`](node).

## Quick Start

Prerequisites: a stable Rust toolchain (`cargo`), Node.js >= 22.13.0, and pnpm
(the pinned version in `node/package.json` is picked up via corepack).

Rust:

```sh
cargo check -p ferriki-core
```

Node (`build:native` compiles the Rust core and copies the addon into the
package):

```sh
cd node
pnpm install --ignore-scripts
pnpm run prepare:compat
pnpm run build:native
pnpm run test:ferriki-compat:textmate
```

The compatibility workspace contains upstream `prepare` scripts that rewrite
generated files. Installs intentionally skip package lifecycle scripts; the
explicit `prepare:compat` command runs those generators in a temporary
checkout and keeps the tracked mirror immutable.

The TextMate compatibility lane runs selected, unchanged Shiki tests with
honest aliases so every highlighted result passes through the native addon.
The broader `test:ferriki-compat:core` lane also runs the Ferriki-owned API,
error, catalog, registration, multi-theme, transformer, token-state, packaging,
and Ferromark/Ardo contract checks. Upstream cases outside that accepted
boundary remain explicitly classified in the compatibility manifest.

## Compatibility

Ferriki tracks one approved Shiki release tag at a time — currently
**Shiki v4.4.3** (machine-readable source of truth:
[`node/compat/upstream/shiki/.source.json`](node/compat/upstream/shiki/.source.json)).

- Upstream-derived files under [`node/compat/upstream/shiki`]node/compat/upstream/shiki are mirrored, not hand-edited.
- vscode-textmate source and tests under [`node/compat/upstream/vscode-textmate`]node/compat/upstream/vscode-textmate are likewise pinned and never hand-edited.
- Ferriki-specific behavior lives outside that mirror, mainly in [`node/compat/harness`]node/compat/harness and the Ferriki product paths.
- “Shiki-compatible” in this repository means compatibility is intended to be checked, not just claimed.
- Compatibility coverage is broader than product scope. Ferriki may still test selected optional upstream adapters separately, but that does not make them first-class Ferriki features.

## Status

The native TextMate runtime from issue #30 is implemented. It is a mechanical
port of pinned vscode-textmate v9.3.2 onto the Scanner API of
[Ferroni](https://github.com/sebastian-software/ferroni), integrated with
Ferriki's asset catalogs, native renderer, N-API host, and focused Node
surface. Its inner vscode-textmate oracle and honest Shiki v4.4.3 structural
gate are green.

The accepted API contract and its focused checks own token/state, lifecycle,
transformer, and registration behavior. Upstream adapter fixtures that do not
describe Ferriki's native boundary remain classified as explicit non-goals.
The current core direction is:

- Rust owns runtime behavior
- Node is the thin facade and compatibility layer
- optional ecosystem adapters do not define the core product boundary or release gate

## License

Licensed under either of [MIT](./LICENSE-MIT) or [Apache-2.0](./LICENSE-APACHE)
at your option.

### <a href="https://ferramenta.dev"><img src="https://raw.githubusercontent.com/sebastian-software/ferramenta/main/app/assets/brand/logo-light.svg" width="24" height="24" alt="" /> More from Ferramenta</a>

[Ferramenta](https://ferramenta.dev) — A family of Rust tools.

**The content pipeline**

| Tool | Job |
| --- | --- |
| [ferroni]https://sebastian-software.github.io/ferroni/ | Oniguruma-compatible regex engine |
| [ferromark]https://sebastian-software.github.io/ferromark/ | Markdown to HTML with a secure default and every GFM extension included. |

**The language workshop**

| Tool | Job |
| --- | --- |
| [ferrolex]https://github.com/sebastian-software/ferrolex | Spell checking for text and code |
| [ferrocat]https://ferrocat.dev | Translation catalog engine |
| [palamedes]https://palamedes.dev | Internationalization for TypeScript applications |

**On the workbench**

| Tool | Job |
| --- | --- |
| [ferrovia]https://github.com/sebastian-software/ferrovia | SVGO-compatible SVG optimizer |
| [ferralk]https://github.com/sebastian-software/ferralk | Glob matching and parallel filesystem walking |
| [ferrugo]https://github.com/sebastian-software/ferrugo | PDF previews for untrusted files |

---

<p align="center">
  <a href="https://oss.sebastian-software.com"><img src="https://raw.githubusercontent.com/sebastian-software/oss.sebastian-software.com/main/app/assets/logo-software.svg" alt="Sebastian Software" width="160" /></a><br />
  TypeScript, React &amp; Rust consulting<br />
  Experts in Agentic Software Development<br />
  <a href="https://sebastian-software.de">Work with us</a> · <a href="https://oss.sebastian-software.com">More open source</a>
</p>

<p align="center">Copyright &copy; 2026 Sebastian Software GmbH</p>