willbooster-tree-sitter-cpp 1.2.2

C++ grammar for tree-sitter
Documentation

@willbooster/tree-sitter-cpp

npm version license Test Test rust semantic-release wbfy crates.io

C++ grammar for tree-sitter, forked from tree-sitter/tree-sitter-cpp. We are grateful to its authors and contributors. This is not an official release of that project.

This fork fixes parsing bugs and raises conformance with the ISO C++ standard (working draft).

Usage

The npm package ships tree-sitter-cpp.wasm for @willbooster/web-tree-sitter, which runs in Node.js, Bun, browsers, and Cloudflare Workers. Install both:

npm install @willbooster/tree-sitter-cpp @willbooster/web-tree-sitter

In Node.js and Bun, load the .wasm file from its path:

import { fileURLToPath } from 'node:url';
import { Language, Parser } from '@willbooster/web-tree-sitter';

await Parser.init();
const parser = new Parser();
const wasmPath = fileURLToPath(import.meta.resolve('@willbooster/tree-sitter-cpp/tree-sitter-cpp.wasm'));
parser.setLanguage(await Language.load(wasmPath));
const tree = parser.parse('int main() {}\n');

In browsers, serve both .wasm files and load them by URL. With Vite:

import { Language, Parser } from '@willbooster/web-tree-sitter';
import runtimeUrl from '@willbooster/web-tree-sitter/web-tree-sitter.wasm?url';
import cppUrl from '@willbooster/tree-sitter-cpp/tree-sitter-cpp.wasm?url';

await Parser.init({ locateFile: () => runtimeUrl });
const parser = new Parser();
parser.setLanguage(await Language.load(cppUrl));

In Cloudflare Workers, which do not allow compiling Wasm at run time, import both .wasm files as modules:

import { Language, Parser } from '@willbooster/web-tree-sitter';
import runtime from '@willbooster/web-tree-sitter/web-tree-sitter.wasm';
import cpp from '@willbooster/tree-sitter-cpp/tree-sitter-cpp.wasm';

await Parser.init({ wasmModule: runtime });
const parser = new Parser();
parser.setLanguage(await Language.load(cpp));

The package also ships the node types in src/node-types.json.

In Rust, depend on the crate and on willbooster-tree-sitter, the runtime this package is tested and fuzzed with (the grammar also loads in the upstream tree-sitter crate 0.27, whose error recovery never ends on some malformed input):

[dependencies]
tree-sitter = { package = "willbooster-tree-sitter", version = "1" }
tree-sitter-cpp = { package = "willbooster-tree-sitter-cpp", version = "1" }

let mut parser = tree_sitter::Parser::new();
parser.set_language(&tree_sitter_cpp::LANGUAGE.into())?;

Development

mise install
bun install --frozen-lockfile
bun run build/ci
bun run test
script/parse-examples
cargo test --locked

bun run test runs:

  • the corpus in test/corpus, with the native build and with the Wasm build (the first run downloads the WASI SDK);
  • an incremental-parsing check (test/unit/incremental.test.ts): tree-sitter fuzz edits each corpus case at random, reparses it, undoes the edits, and reparses again. TREE_SITTER_SEED, TREE_SITTER_ITERATIONS, and TREE_SITTER_EDITS run other or more edits;
  • a check that the real-world C++ files in examples/, the checked-in ones and those of the cloned repositories, fail to parse exactly as listed in script/known-failures.txt. The first run clones the repositories. The example repositories are pinned to commits in script/parse-examples. After a grammar change or a moved pin alters that list, script/parse-examples rewrites it; review its diff before committing;
  • a performance check (test/unit/performance.test.ts) that recovering from an error on each line takes linear time, since consumers parse files while they are being edited. It loads the Wasm build through @willbooster/web-tree-sitter, which bun run build/ci rebuilds after regenerating the parser;
  • a check (test/unit/expressionKeywordParameter.test.ts) that every keyword that can start an expression but not a parameter is expected in the parameters of the function declarators that compete with a direct initialization, so that long(n)(sizeof(b)); stays an initialization;
  • a check (test/unit/runtimeVersion.test.ts) that @willbooster/web-tree-sitter in package.json and willbooster-tree-sitter in Cargo.lock are the same version, since the Wasm tests run on the former and the Rust tests and the fuzzer on the latter;
  • checks that the Wasm build parses in Chromium (test/unit/browser/) and in Cloudflare Workers with and without Node.js compatibility (test/unit/workers.test.ts). Run bun run test/ci-setup once to install Chromium.

CI also runs these tests on Linux arm64 and macOS, where the parser and scanner are compiled natively against each platform's C library, and fuzzes the parser with libFuzzer and sanitizers (.github/workflows/robustness.yml).

References