willbooster-tree-sitter-javascript 1.0.5

JavaScript grammar for tree-sitter
Documentation

@willbooster/tree-sitter-javascript

npm version license Test Test rust semantic-release wbfy crates.io

JavaScript and JSX grammar for tree-sitter, forked from tree-sitter/tree-sitter-javascript. We are grateful to its authors and contributors. This is not an official release of that project.

This fork fixes parsing bugs and raises conformance with ECMAScript (ECMA-262) and JSX.

Usage

The npm package ships tree-sitter-javascript.wasm for @willbooster/web-tree-sitter, which runs in Node.js, Bun, browsers, and Cloudflare Workers. In Node.js and Bun, load it from the package:

import { fileURLToPath } from 'node:url';
import { Language, Parser } from '@willbooster/web-tree-sitter';

await Parser.init();
const parser = new Parser();
const wasmPath = fileURLToPath(import.meta.resolve('@willbooster/tree-sitter-javascript/tree-sitter-javascript.wasm'));
parser.setLanguage(await Language.load(wasmPath));
const tree = parser.parse('const App = () => <p>Hello</p>;\n');

In browsers, let your bundler serve both .wasm files and pass their URLs:

import { Language, Parser } from '@willbooster/web-tree-sitter';
import runtimeUrl from '@willbooster/web-tree-sitter/web-tree-sitter.wasm?url'; // Vite
import javascriptUrl from '@willbooster/tree-sitter-javascript/tree-sitter-javascript.wasm?url';

await Parser.init({ locateFile: () => runtimeUrl });
const parser = new Parser();
parser.setLanguage(await Language.load(javascriptUrl));

In Cloudflare Workers, which do not allow compiling Wasm at run time, import both .wasm files as modules, with or without Node.js compatibility:

import { Language, Parser } from '@willbooster/web-tree-sitter';
import runtime from '@willbooster/web-tree-sitter/web-tree-sitter.wasm';
import javascript from '@willbooster/tree-sitter-javascript/tree-sitter-javascript.wasm';

await Parser.init({ wasmModule: runtime });
const parser = new Parser();
parser.setLanguage(await Language.load(javascript));

The package also ships the node types in src/node-types.json, and grammar.js and the queries in queries/ for grammars that extend this one (e.g. require('@willbooster/tree-sitter-javascript/grammar')).

In Rust, depend on the crate and on willbooster-tree-sitter, the runtime this package is tested and fuzzed with (the grammar also loads in the upstream tree-sitter crate 0.27, whose error recovery never ends on some malformed input):

[dependencies]
tree-sitter = { package = "willbooster-tree-sitter", version = "1" }
tree-sitter-javascript = { package = "willbooster-tree-sitter-javascript", version = "1" }

let mut parser = tree_sitter::Parser::new();
parser.set_language(&tree_sitter_javascript::LANGUAGE.into())?;

Development

mise install
bun install --frozen-lockfile
bun run build/ci
bun run test
script/parse-examples
cargo test

bun run test runs:

  • the corpus in test/corpus, with the native build and with the Wasm build (the first run downloads the WASI SDK);
  • an incremental-parsing check (test/unit/incremental.test.ts): tree-sitter fuzz edits each corpus case at random, reparses it, undoes the edits, and reparses again. TREE_SITTER_SEED, TREE_SITTER_ITERATIONS, and TREE_SITTER_EDITS run other or more edits;
  • a check that the real-world JavaScript files in examples/, the checked-in ones and those of the cloned repository, fail to parse exactly as listed in script/known-failures.txt. The first run clones the repository. The example repository is pinned to a commit in script/parse-examples. After a grammar change or a moved pin alters that list, script/parse-examples rewrites it; review its diff before committing;
  • a performance check (test/unit/performance.test.ts) that recovering from an error on each of 10,000 lines takes linear time, since consumers parse files while they are being edited. It loads the Wasm build through @willbooster/web-tree-sitter, which bun run build/ci rebuilds after regenerating the parser;
  • checks that the Wasm build parses in Chromium (test/unit/browser/) and in Cloudflare Workers with and without Node.js compatibility (test/unit/workers.test.ts). Run bun run test/ci-setup once before the first run to install Chromium and, on Linux, the system libraries it needs.

CI also runs these tests on Linux arm64 and macOS, where the Rust binding compiles the parser natively, and fuzzes the parser with libFuzzer and sanitizers (.github/workflows/robustness.yml).

References