bcgen-lang 0.2.0

Compiles HIR to LSB bytecode: the path from a forged language's meaning to code bvm-lang runs.
Documentation

The model

  • compile takes a resolved hir_lang::Hir and the interner its names came from, and returns a Compiled: the LSB module, its entry function, and an origin table mapping every instruction to its HIR node.
  • Compiler adds Options (array semantics, key coercion, what the module body compiles to, hooks, budgets) and a Locate for real line numbers in the line tables (LineIndex for one source file).
  • CompileError says why a unit does not compile, naming the HIR node and its span; Unsupported names the forms planned for later releases.

What it guarantees, and how each guarantee is checked:

Guarantee How it is held
Compiled programs mean what the HIR means. tests/differential.rs: random well-formed programs (functions, closures, loops, match, try/finally, defer, maps, every op under random policies, promote included) run on bvm-lang and on a reference HIR interpreter written in the tests; results, printed output, error kinds, and error locations (the raising HIR node) must agree, under both array profiles. 1,000,000 programs were run before this release; every cargo test runs 3,000 more. Mutation-checked: removing the array copy, swapping two finally dispatch targets, or compiling floor_mod as rem makes it fail.
Every module it emits loads in bvm-lang. tests/properties.rs: every generated program's module passes bvm-lang's load-time checks (registers, call windows, constants, branch targets, handlers in range) and round-trips through the LSB encoding unchanged.
Every instruction maps back to its HIR origin. The origin table holds one node per instruction of every function, each a node of the unit; line-table rows are strictly increasing; a runtime error's location maps to the raising node and, through a LineIndex, to its source line (tests/properties.rs).
Compilation is deterministic. The same unit compiles to the same bytes, from a clone and on another thread (tests/properties.rs).
No recursion over the input. Code generation is an explicit task stack: 100,000 nested operations and 20,000 nested ifs inside 20,000 blocks compile on a 512 KiB stack in a debug build (tests/limits.rs). Output is bounded by an instruction budget.
PHP array semantics. Arrays are values (copied on assignment, passing, returning, capture, and store, with an O(1) copy-on-write dup), nested writes separate shared arrays and create missing ones, and keys follow PHP's coercion, checked against a reference interpreter that makes real deep copies, and by tests/keys.rs against PHP's canonical-integer rule.
A Mox program runs end to end. tests/e2e.rs: a hand-lowered Mox program is resolved by resolve-lang, compiled, and run on bvm-lang, printing through a host function.

Installation

[dependencies]

bcgen-lang = "0.2"

Without the standard library:

[dependencies]

bcgen-lang = { version = "0.2", default-features = false }

Quick start

Compile a unit and run it on bvm-lang:

use bvm_lang::{Host, Program, Value, Vm};
use hir_lang::{Builder, Name, OpKind};
use intern_lang::Interner;

// fn add(a, b) { a + b }
let mut names = Interner::new();
let mut b = Builder::new();
let (pa, a) = b.local_param(Name::new(names.intern("a")));
let (pb, bb) = b.local_param(Name::new(names.intern("b")));
let (ua, ub) = (b.use_binder(a), b.use_binder(bb));
let sum = b.op(OpKind::Add, &[ua, ub]);
let body = b.block(&[], Some(sum));
let add = b.func(Name::new(names.intern("add")), &[pa, pb], body);
let root = b.module(None, &[add]);
let hir = b.finish(root)?;

let out = bcgen_lang::compile(&hir, &names)?;
let func = out.function(add).unwrap();
let program = Program::load(out.into_module(), &Host::new())?;
let mut vm = Vm::new(&program);
assert_eq!(vm.run(func, &[Value::Int(2), Value::Int(3)])?, Value::Int(5));
# Ok::<(), Box<dyn std::error::Error>>(())

Scripts, host functions, and source lines

A module body runs in the module's start function and the entry function returns its value. Host symbols (Res::Extern) become LSB imports:

use bcgen_lang::{Compiler, LineIndex, Options};
use bvm_lang::{Host, Program, Value, Vm};
use hir_lang::{Builder, Effects, Expr, Item, ItemKind, List, Name, Ns, Path, Res, Segment};
use intern_lang::Interner;

// echo("hello"); 42
let mut names = Interner::new();
let mut b = Builder::new();
let echo = names.intern("echo");
let seg = Segment::new(Name::new(echo), b.origin());
let segs = b.list(&[seg]);
let path = b.path(Path { res: Res::Extern(echo), ..Path::new(segs, Ns::Value) });
let callee = b.expr(Expr::Path(path));
let hello = b.str_lit("hello");
let call = b.call(callee, &[hello]);
let stmt = b.expr_stmt(call);
let answer = b.int(42);
let body = b.block(&[stmt], Some(answer));
let module = ItemKind::Module { items: List::EMPTY, body: Some(body), effects: Effects::NONE };
let root = b.item(Item::new(None, module));
let hir = b.finish(root)?;

let lines = LineIndex::new("hello.mox", 0, "echo(\"hello\"); 42\n");
let out = Compiler::new(Options::new()).with_locator(&lines).compile(&hir, &names)?;

let mut host = Host::new();
host.register("host", "echo", |ctx, args| {
    assert_eq!(ctx.str_bytes(args[0]), Some(&b"hello"[..]));
    Ok(Value::Nil)
});
let program = Program::load(out.module().clone(), &host)?;
assert_eq!(Vm::new(&program).run(out.entry(), &[])?, Value::Int(42));
# Ok::<(), Box<dyn std::error::Error>>(())

The full reference, including the calling convention, the dynamic semantics of every form, and what each option changes, is in docs/API.md.

Performance

Release build on the development machine (Windows 11), criterion medians from benches/bench.rs. Compile figures are library-only (HIR in memory to an LSB module, no encoding); run figures are bvm-lang 2.0.0-alpha.1 executing the compiled module (VM instance creation included).

Benchmark Result
Compile, synthetic unit, 152k HIR nodes 23 ms (6.6 M nodes/s)
Compile, synthetic unit, 532k HIR nodes 103 ms (5.2 M nodes/s)
Compile, synthetic unit, 1.52M HIR nodes 354 ms (4.3 M nodes/s)
fib(25), direct calls 8.6 ms
Counting loop, 1M iterations 10.7 ms
PHP array: 100k appends, then 100k indexed reads 13.0 ms
The same plus 100k nested writes g[i % 64][] = i 371 ms

"Nodes" counts items, expressions, statements, patterns, paths, and parameters. Compile time is linear: tests/scaling.rs measures 150 to 170 ns per node from 146k to 2.3M nodes (the criterion spread above is allocator noise around that). The last row is the cost of PHP nested-write separation: LSB cannot ask whether an inner array is shared, so every nested write copies the inner array's store while the old copy is uncollected (about 28 times the plain-append case here). An LSB instruction that separates only when shared is requested from the spec owners.

Contributing

See dev/DIRECTIVES.md for engineering standards and the definition of done. Before a PR: cargo fmt --all, cargo clippy --all-targets --all-features -- -D warnings, and cargo test --all-features must be clean.