Crate lang_forge 

lang_forge

LexerSketch: forge a working language front end — lexer, parser, and lossless syntax tree — from a .lsf schematic.

A schematic is a short NOML document that describes a language: its name, how its tokens look, its grammar, and the capabilities (passes) it includes. Language::from_lsf reads it, checks it, and compiles it into tables; Language::parse then turns source text into a syntax_lang::Node tree with diag_lang::Diagnostics for anything malformed. There is no code generation step and nothing to build: the language is ready the moment the schematic is forged.

lang-forge is the capstone of the -lang language-construction family. Its trees are the family’s lossless CST, so a forged language plugs straight into the formatter, incremental reparser, language server, and tree-sitter crates; its diagnostics render with diag-lang; and its capabilities run on pass-lang.

A first language

use lang_forge::Language;

let calc = Language::from_lsf(
    r##"
    [language]
    name = "calc"

    [lexer]
    line_comments = ["#"]

    [rules]
    program = "stmt*"
    stmt    = "'let' IDENT '=' expr ';' | expr ';'"
    group   = "'(' expr ')'"

    [rules.expr]
    operand = "NUMBER | IDENT | group"
    levels  = [
        { left   = ["+", "-"] },
        { left   = ["*", "/"] },
        { prefix = ["-"] },
        { right  = ["^"] },
    ]
    "##,
)?;

let parse = calc.parse("let area = 3 * r ^ 2; # circle-ish\n");
assert!(!parse.has_errors());

// `^` binds tighter than `*`, so the product's right operand is `r ^ 2`.
let binary = calc.kind("binary").expect("the default operator node");
let product = parse.tree().descendants().find(|n| *n.kind() == binary).expect("3 * r ^ 2");
assert_eq!(product.text(parse.source()), Some("3 * r ^ 2"));
assert_eq!(product.child_nodes().last().and_then(|n| n.text(parse.source())), Some("r ^ 2"));

The rule language

Each entry of [rules] is a rule. A string rule is a sequence of elements:

Element	Matches
'text' or "text"	a keyword (if it looks like an identifier) or a symbol
IDENT, NUMBER, STRING, NEWLINE	a token of that built-in class
name	the rule name, as a child node
a b	a then b
a | b	a, or else b — the first that matches wins
a*, a+, a?	zero or more, one or more, zero or one
( ... )	grouping

A rule builds a node named after itself, unless its name starts with _, in which case its children are placed directly in the parent. The first rule is the start rule unless [language] start names another; its node is the root of every tree.

A table rule, [rules.name], is an expression rule: an operand and operator levels, lowest precedence first. Each level is left, right, or none (binary, by associativity), prefix, or postfix, with an optional then (more grammar after the operator, for calls, indexing, or ?:) and an optional node name.

Errors and recovery

Forging reports every problem in a schematic at once, each with a span into the schematic (Error). Parsing never fails: a missing token is reported and assumed, an unexpected token is reported and wrapped in an ERROR node, and the tree always covers the whole source.

Features

• std (default) — the standard library. Without it the crate is no_std and needs only alloc.

Re-exports

syntax_lang, diag_lang, and pass_lang are re-exported whole, so code that walks trees, renders diagnostics, or writes capability passes names the same versions lang-forge was built against.

Stability

The public surface is frozen as of 1.0.0 and follows Semantic Versioning: no breaking change before 2.0, additions arrive in minor releases, and the MSRV (Rust 1.85) only rises in a minor. The promise covers the API, the .lsf schematic format, the trees forged languages build from valid input, and the parser’s guarantees; it is set out in full in docs/API.md.

Re-exports

pub use diag_lang;

pub use pass_lang;

pub use syntax_lang;

Structs

Error

Why a schematic could not be forged into a Language, or a capability pipeline could not be assembled.

Kind

The kind of a token or node in a forged language’s syntax tree.

Language

A language forged from a .lsf schematic: a lexer, a parser, and the kinds of its syntax tree.

Parse

The result of Language::parse: a lossless syntax tree and the problems found while building it.

Type Aliases

Capability

A capability pass, boxed for Language::pipeline.