pd-vm 0.22.4

RustScript bytecode compiler and VM
Documentation

RustScript

rustscript on crates.io

RustScript is the language, VM/compiler core, standard library, examples, bytecode/AOT tooling, wasm runtime support, and debugger-facing runtime contract for the RustScript family. Compatibility language frontends now live in source-plugin crates.

Related projects

Crate usage

Consumers can refer to the VM crate from this repository:

rustscript = "0.22.2"
pd-vm = { git = "https://github.com/rustscript-lang/rustscript", package = "pd-vm" }
pd-host-function = { git = "https://github.com/rustscript-lang/rustscript", package = "pd-host-function" }

Test

cargo test --workspace
cargo build --workspace --release

pd-vm is a stack-based virtual machine plus compiler toolchain. It includes the RustScript (.rss) frontend and exposes a source-plugin API for compatibility languages such as JavaScript and Lua.

Contents

Overview

Executes compiled compact bytecode rather than interpreting source. Offers consistent runtime semantics for both synchronous and asynchronous execution. Includes rich debugging and profiling tools: interactive debugger, recording and replay, and JIT trace insights. Emits compile-time type metadata that the interpreter and trace JIT use for typed fast paths and clearer compile diagnostics. RustScript now treats that metadata as required compiler output rather than optional hints.

TODO

  • Rust-like Option/Result support.
  • Epoch check should be only on loop start or function call, instead of per interval.
  • host call fuel budgeting.
  • Callable-as-value support.

How To Use

Run Programs

Run with the VM runner binary:

cargo run -p pd-vm --bin pd-vm-run -- --fuel 100000 examples/example.rss

Compatibility frontends such as JavaScript and Lua are provided by source-plugin crates. Use CompileSourceFileOptions::with_source_plugin(...) when compiling those files from an embedding crate.

REPL

RustScript REPL (history + multiline support):

cargo run -p pd-vm --bin pd-vm-run -- --repl

Debugging

Run with interactive pdb debugger on stdio:

cargo run -p pd-vm --bin pd-vm-run -- --debug examples/example.rss

Run debugger over TCP:

cargo run -p pd-vm --bin pd-vm-run -- --debug --tcp 127.0.0.1:9002 examples/example.rss

Useful commands: break, break line, step, next, out, stack, locals, where, continue, fuel, epoch.

Recording and Replay

Record execution:

cargo run -p pd-vm --bin pd-vm-run -- --record out/example.pdr examples/example.rss

Replay execution:

cargo run -p pd-vm --bin pd-vm-run -- --view-record out/example.pdr

Replay supports break, break line, continue, step, next, out, stack, locals, print, ip, where, and funcs. In replay mode, breakpoints set pause points in the replay stream instead of runtime VM breakpoints.

Bytecode and VMBC

Emit VMBC wire-format output without running:

cargo run -p pd-vm --bin pd-vm-run -- --emit-vmbc out/example.vmbc examples/example.rss

Disassemble VMBC:

cargo run -p pd-vm --bin pd-vm-run -- --disasm-vmbc path/to/program.vmbc

Disassemble with embedded source (if present):

cargo run -p pd-vm --bin pd-vm-run -- --disasm-vmbc path/to/program.vmbc --show-source

JIT

Dump trace-JIT activity:

cargo run -p pd-vm --bin pd-vm-run -- --jit-hot-loop 2 --jit-dump examples/example.rss

Native JIT codegen uses Cranelift.

  • Cranelift is part of the Bytecode Alliance/Wasmtime ecosystem
  • NYI behavior is shared by the trace recorder (TraceJitEngine) and is backend-independent.
  • Some operations may use helper fallback paths internally, but that is not counted as trace-recorder NYI.
  • Backward brfalse can stay inside a trace as a LoopIfFalse back-edge when the target already exists in the recorded trace; backward targets outside the trace still become side exits.

Library hooks:

  • vm.set_jit_config(...)
  • vm.jit_snapshot()
  • vm.dump_jit_info()
  • vm.jit_native_trace_count()
  • vm.jit_native_exec_count()

Fuel Metering

pd-vm provides Wasmtime-style fuel controls on both Vm and Store<T>:

  • set_fuel
  • set_fuel_check_interval
  • fuel_check_interval
  • get_fuel
  • consume_fuel
  • consume_fuel_tick
  • add_fuel / recharge_fuel (Store::recharge)
  • fuel_checkpoint / checkpoint
  • restore_fuel / restore_checkpoint

Store<T> is a lightweight wrapper around Vm plus host context data (data() / data_mut()), and forwards run() / resume().

pd-vm-run supports --fuel <n> to set the initial VM fuel budget. pd-vm-run also supports --epoch-deadline <n> plus --epoch-check-interval <n> for Wasmtime-style epoch interruption.

Debugger fuel commands:

  • fuel (show remaining fuel and check interval)
  • fuel set <n>
  • fuel add <n>
  • fuel clear
  • fuel interval [n]

Example:

use vm::{Store, VmStatus};

// ... create vm ...
let mut store = Store::from_vm(vm);
store.set_fuel(10_000);
store.set_fuel_check_interval(1)?; // exact mode: check every instruction/trace op
let checkpoint = store.checkpoint();

loop {
    match store.run()? {
        VmStatus::Halted => break,
        VmStatus::Yielded => continue,
        VmStatus::Waiting(_) => {
            store.vm_mut().wait_for_host_op_blocking()?;
        }
    }
}

store.recharge(1_000)?;
store.restore_checkpoint(checkpoint);

Fuel charging semantics:

  • Fuel metering is disabled by default (get_fuel() == None).
  • Fuel metering and epoch interruption are mutually exclusive. Enabling one disables or rejects the other API surface, depending on the operation.
  • set_fuel sets an explicit budget; add_fuel also enables metering if it was disabled.
  • Fuel is consumed in chunks at the configured check cadence. Chunk size = fuel_check_interval. Default interval is 1 (exact mode).
  • The interpreter applies fuel checks in the VM loop before opcode fetch/execute.
  • Trace-JIT execution applies the same cadence against recorded trace ops/blocks.
  • When fuel metering is enabled, native JIT execution injects fuel checks in generated machine code at the configured check cadence.
  • With interval > 1, out-of-fuel detection is coarse-grained: execution may run up to interval - 1 extra instructions before the next fuel check.
  • If there is not enough fuel, execution returns VmStatus::Yielded before the next instruction runs (instruction pointer is not advanced). Top up fuel (set_fuel / add_fuel) and call run() / resume() again.
  • FuelCheckpoint snapshots only fuel-accounting state (remaining budget, check interval, and current check-phase cursor). Restoring a checkpoint does not rewind VM stack, locals, or instruction pointer.
  • Host-side work is not automatically metered beyond VM instruction execution; host code can call consume_fuel explicitly for additional charging policy.

Epoch Interruption

pd-vm also provides a Wasmtime-style epoch API:

  • epoch_handle
  • current_epoch
  • increment_epoch / increment_epoch_by
  • set_epoch_deadline
  • clear_epoch_deadline
  • epoch_deadline
  • set_epoch_check_interval
  • epoch_check_interval
  • epoch_checkpoint / restore_epoch
  • last_yield_reason

Semantics:

  • EpochHandle is shared engine-style state. Callers advance it externally.
  • The epoch unit is an abstract tick, not wall-clock time by itself.
  • set_epoch_deadline(n) arms the VM to yield once current_epoch >= current_epoch_at_arm + n.
  • Epoch interruption reuses VmStatus::Yielded; inspect last_yield_reason() if the caller needs to distinguish fuel vs epoch vs host yields.
  • After an epoch yield, the next run() / resume() automatically re-arms the same deadline delta. Use epoch deadline <n> to change the slice size or epoch clear to disable interruption.
  • The interpreter and native JIT use the same inline checkpoint cadence (epoch_check_interval).

Debugger epoch commands:

  • epoch (show current epoch, deadline, and check interval)
  • epoch tick [n]
  • epoch deadline <n>
  • epoch clear
  • epoch interval [n]

Wasm Lint

Compiler-only wasm build (without runtime/JIT/debugger/CLI):

cargo check -p pd-vm --target wasm32-unknown-unknown --no-default-features

Browser/editor lint integration is provided by sibling crate pd-vm-wasm via lint_source_json.

The wasm linter reports both parse errors and compile-time type errors with Monaco-friendly line and span metadata. That includes inferred-type failures such as incompatible if/else branch merges.

Wasm Runtime Playground

Runtime-enabled wasm build (without native JIT backend):

cargo check -p pd-vm --target wasm32-unknown-unknown --no-default-features --features runtime

Browser playground wasm runtime is provided by sibling crate pd-vm-wasm built with the runtime feature via:

  • lint_source_json
  • run_source_json

no_std Embedded Runtime

The sibling crate pd-vm-nostd provides the VMBC v8 decoder and compact interpreter using only core and alloc. It supports direct bytecode execution, synchronous host callbacks, and instruction fuel while leaving source compilation, CLI, debugger, JIT/AOT, async host operations, and operating-system integrations in pd-vm.

RP2040 compile check:

rustup target add thumbv6m-none-eabi
cargo check -p pd-vm-nostd --target thumbv6m-none-eabi

Compile .rss source to VMBC on a host, then load that VMBC through pd_vm_nostd on the target. The PlatformIO Arduino-Pico integration lives in micro-rustscript.

Web Playground

The Monaco browser playground lives in a standalone repository:

It builds pd-vm-wasm --features runtime, copies wasm artifacts into the web app, and uses the Monaco grammar assets kept under editor-assets/monaco.

In browser epoch mode, the playground drives one epoch tick from a 1ms JavaScript timer and shows the live epoch counter in the interruption panel. Timer delivery still depends on the main thread, so compute-only wasm cannot be preempted mid-call while the browser is busy running the VM.

Test and Perf Commands

Integration example tests:

cargo test -p pd-vm --test example_tests

Manual perf characterization (ignored by default):

cargo test -p pd-vm --test jit_tests perf_ -- --ignored --nocapture

Migration perf record (handwritten vs Cranelift baseline before handwritten backend removal):

Internals

VM Internals

Program consists of:

  • constants: Vec<Value>
  • code: Vec<u8>
  • local_count: usize
  • imports: Vec<HostImport>
  • debug: Option<DebugInfo>
  • type_map: Option<TypeMap>

TypeMap currently contains:

  • operand_types: HashMap<usize, (ValueType, ValueType)>
  • local_types: Vec<ValueType>

operand_types is keyed by bytecode offset and records inferred operand pairs for emitted binary ops/comparisons. The interpreter and trace recorder use it for typed int/float/string fast paths. local_types is a post-lowering local-slot summary and is intentionally lossy: compiler-inserted null clears and hidden frame slots can widen entries to Unknown/Null, so it should not be treated as a source-level binding type table.

Bytecode format:

  • 1 byte opcode
  • little-endian operands (u8, u16, u32)
  • absolute jump targets

Instruction set:

Opcode Mnemonic Operands Stack effect
0x00 nop - no change
0x01 ret - stop execution
0x02 ldc u32 index push constant
0x03 add - (a, b) -> (a + b)
0x04 sub - (a, b) -> (a - b)
0x05 mul - (a, b) -> (a * b)
0x06 div - (a, b) -> (a / b)
0x07 neg - (a) -> (-a)
0x08 ceq - (a, b) -> (a == b)
0x09 clt - (a, b) -> (a < b)
0x0A cgt - (a, b) -> (a > b)
0x0B br u32 target ip = target
0x0C brfalse u32 target pop bool; if false jump
0x0D pop - pop value
0x0E dup - dup top of stack
0x0F ldloc u8 index push local
0x10 stloc u8 index pop -> local
0x11 call u16 id, u8 argc pop args, call host, push returns
0x12 shl - (a, b) -> (a << b)
0x13 shr - (a, b) -> (a >> b)
0x14 mod - (a, b) -> (a % b)
0x15 and - (a, b) -> (a && b)
0x16 or - (a, b) -> (a || b)

Host calls and resuming:

The call opcode pops its arguments, dispatches to a builtin or bound host function via Vm::execute_host_call, and handles three distinct outcomes:

CallOutcome Meaning IP after suspension Stack after suspension
Return(values) Synchronous result Advanced past call Return values pushed
Yield Retry next run() Rewound to call opcode Args re-pushed
Pending(op_id) Async result pending Advanced past call Empty (result injected via complete_host_op)

CallOutcome::Yield — cooperative "retry me later":

  • The VM re-pushes the original args onto the stack and rewinds self.ip to the start of the call opcode (call_ip). The next run() / resume() re-executes the entire call from scratch, re-popping args and re-invoking the host function.
  • The host function must be idempotent with respect to yield (it will be called again).
  • Returns VmStatus::Yielded with last_yield_reason() == Some(VmYieldReason::Host).

CallOutcome::Pending(op_id) — async host operation:

  • The VM advances self.ip past the call instruction (to call_ip + 4) and records waiting_host_op = Some(op_id).
  • Subsequent calls to run() immediately return VmStatus::Waiting(op_id) until the caller resolves the op via vm.complete_host_op(op_id, values) (which pushes return values) or via vm.poll_waiting_host_op() / vm.await_waiting_host_op().

Cooperative interruption yields (fuel / epoch):

  • The fuel/epoch check fires in the interpreter loop before each opcode fetch, so no instruction is partially executed when a budget yield occurs. The IP is unchanged; refueling and calling run() again resumes from exactly the same instruction.
  • A special case exists for the fused call; ret tail pattern: when the call immediately precedes ret, the VM consumes the trailing ret inline and charges an extra interrupt tick. If that extra tick fires an out-of-fuel/epoch error, it is caught and surfaced as VmStatus::Yielded after the call already completed and return values are on the stack. The IP at that point is past the ret, so the resumed run() halts cleanly.

Context-switch safety invariant:

There are two distinct suspension categories, each safe for a different reason:

  1. Host-driven suspension (CallOutcome::Yield and CallOutcome::Pending) is only triggered from within execute_host_call, which is only reachable via the call opcode handler. Every other instruction either completes fully or returns a hard VmError, so host-driven suspension never interrupts a partially-executed instruction.

  2. Cooperative interruption (fuel / epoch) fires at any instruction boundary — the check runs in the interpreter loop before the next opcode is fetched, so the VM can pause before an assignment, an arithmetic op, a branch, etc. This is safe because no instruction has started executing: the IP points at the unconsumed opcode and the stack is in a fully consistent state from the previous instruction's completion.

In both cases there is never a partially-executed instruction left in flight when the VM suspends. If a new instruction or path ever needs to trigger a suspension, it must ensure the stack and IP are fully coherent before doing so.

Compiler Internals

Pipeline Layers

The end-to-end stack is split into layers. Not every entrypoint uses every layer (for example, compile_source() skips module loading/linking), but this is the full model:

  1. Module/source loading (compile_source_file() path)
  2. Unit linking (linker::merge_units)
  3. Frontend lowering (built-in rustscript, plus any registered source plugins)
  4. Frontend-independent IR
  5. Type-consistency validation on legalized IR (for example rejecting known if/else branch mismatches)
  6. Lifetime/liveness lowering plus type metadata collection
  7. Bytecode backend (Compiler + Assembler -> Program) executed by VM
  8. Trace-JIT SSA recording (JitTrace + SsaTrace) with symbolic stack/local state
  9. Native machine code emission and execution from SSA traces

Type Metadata and Inference

The compiler runs lightweight value-type inference before bytecode emission and records the result into Program.type_map when metadata is available.

Current important behaviors:

  • Known arithmetic on int/int stays int.
  • Known mixed numeric arithmetic widens to float.
  • + becomes string concatenation when either side is known string, so cases such as "text" + 123 are recorded as string-concat operand metadata.
  • Callable return types propagate through direct named calls, function-valued locals, closure-valued locals, and callable parameters when the callee can still be identified.
  • if/else expression results and branch-local merges are rejected when both sides have different known concrete types. The compiler no longer keeps backward-compatible dynamic fallback behavior for those mismatches.

This inference is intentionally local and pragmatic. It exists to drive compile diagnostics and monomorphic runtime fast paths, not to provide a full source-language static type system.

Compiler APIs

Use compile_source() for RustScript, or compile_source_file() for built-in .rss path loading. For compatibility languages, build options with CompileSourceFileOptions::with_source_plugin(...) and call compile_source_file_with_options(...).

fn print(x);
let x = 2 + 3;
let y = x * 4;
if y > 10 {
    print(y);
} else {
    0;
}

Closure subset example:

let base = 7;
let add = |value| value + base;
add(5);

Compatibility frontends can lower equivalent closure forms through the source-plugin API.

Built-in print aliases (no declaration needed):

  • RustScript: print(value);, print("... {}", a);, println(value);, println("... {}", a);
  • Compatibility frontends may provide their own print aliases through plugin lowering.

Host calls must be explicitly imported:

  • RustScript: use runtime;, use http;, use rate_limit;
  • Compatibility frontends own their import syntax through SourcePlugin::parse_imports(...) and SourcePlugin::strip_imports(...).

Assembler API

Use assemble() to parse text assembly into a Program.

Data declarations:

  • const NAME VALUE
  • string NAME "..."
.data
const two 2
string greeting "hello"
.code
.local counter
.label loop
ldc two
stloc counter
ldloc counter
ldc 1
sub
dup
stloc counter
brfalse done
br loop
.label done
ldc greeting
ret

Directives:

  • .data and .code switch sections
  • .label NAME defines a jump label
  • .local NAME [INDEX] defines a named local

Builtins and Bridged call Opcode

The compiler uses one call shape (Expr::Call -> OpCode::Call) and distinguishes targets by call index.

  1. Builtin calls (fixed reserved indices)
    • Builtins use BuiltinFunction::call_index()
    • parser lowering emits these for helpers such as len, get, set, slice, count, type_of, assert, and io::*/re::*/json::*/jit::*
  2. Runtime host imports (per-program remapped indices)
    • non-inlined runtime imports are remapped to dense import slots (call_index_remap)
    • emitted as call <slot>, <argc>
    • exposed as Program.imports and bound via HostFunctionRegistry
    • runtime::sleep(ms) is available as a default host import; native runtimes block for the requested duration and wasm runtimes return true immediately
  3. Inlined RustScript function bodies
    • calls to targets with FunctionImpl are inlined (no emitted call)

At runtime, call is bridged through Vm::execute_host_call:

  • builtin call indices dispatch to vm/builtin_runtime.rs
  • non-builtin indices dispatch to bound host imports
  • trace-JIT records supported hot paths into SSA and falls back to the interpreter for call-heavy or otherwise unsupported traces, preserving interpreter semantics

Current Compiler Subset Limitations

Core compiler/IR:

  • callable locals can be passed and called, but callables are not runtime Values
  • callable values cannot currently be stored in arrays/maps or returned from functions
  • callable return-type inference propagates through direct named calls, callable locals, closures, and callable parameters when the compiler can still identify the callee
  • known if/else expression results and branch-local merges with incompatible concrete types are compile errors
  • RustScript uses explicit nullable schemas such as int? and Profile?; non-optional declared locals and returns reject null
  • in RustScript, optional chaining requires a user-declared schema on the container; the result stays optional until handled with .unwrap_or(...), a != null refinement, or a match arm that binds Some(name)
  • after optional handling, the compiler and wasm lint keep the concrete inner type instead of degrading back to unknown
  • recursive RustScript function declarations are not supported by current inlining-based lowering
  • function declarations can be nested and implicitly capture outer locals (closure-like snapshot at declaration time)
  • in RustScript move-semantics mode, implicit captures follow expression semantics (x may move, x.copy() copies, &x/&mut x capture borrowed views)
  • match patterns are limited to int/string/null literals, None, Some(name), _, and type constructors (Some(TypeName))
  • break and continue are only valid inside loops
  • direct host namespace syntax uses named namespaces such as runtime, http, and rate_limit when the corresponding module is not present (builtin namespaces are io::, re::, json::, and jit::)

Module/source loading:

  • crate::... module paths are not supported in RustScript source loading; use relative module paths

Source plugins:

  • Compatibility-language parsing, lowering, and source import scanning belong in plugin crates.
  • pd-vm exposes SourcePlugin, FrontendIr, parser dialect helpers, and IR builder types for plugin authors.
  • compile_source_file() without options only handles built-in .rss; use compile_source_file_with_options() for plugin-backed extensions.

JIT Internals

The VM has two native codegen paths:

  • AOT: lowers the whole bytecode CFG into ahead-of-time native segments.
  • Trace JIT: records hot loops, prefers SSA native lowering when the recorder can specialize the loop, and otherwise leaves the operation on a call/exit path or records NYI.

Trace JIT remains LuaJIT-style hot-loop tracing:

  • hot bytecode loop heads are detected
  • a trace is recorded from each hot root
  • backward brfalse can loop inside the trace when it targets an earlier recorded step
  • native machine code is emitted per compiled trace and invoked by the VM
  • unsupported opcodes or trace shapes fall back to interpreter and are recorded as NYI

Status legend:

  • Inline: dedicated native lowering, no runtime helper/bridge on the fast path.
  • Helper: compiled native code calls a narrow native helper but stays in compiled execution.
  • Bridge/Exit: no dedicated lowering for that backend; execution stays on the normal runtime call path or exits the trace around that operation.
  • NYI: not currently lowered by that backend.

The table below tracks lowering state once typed lowering/specialization is available. For trace JIT, frontend/source patterns may still reach a Bridge/Exit path even when an SSA inline form exists.

Operator family AOT Trace JIT
Typed numeric +, -, *, /, %, unary - Inline Inline
Typed numeric ==, <, > Inline Inline
<< Inline Inline
>>, >>> Inline Inline
Eager and, or, not opcodes Inline Inline
Short-circuit &&, || source forms Lowered as branches, then compiled Lowered as branches, then traced/compiled
Builtin / container op AOT Trace JIT
String concat / bytes concat Inline Inline
len(string), len(bytes) Inline Inline
get(string), get(bytes) Inline Inline
slice(string), slice(bytes) Inline Inline
has(bytes) Inline Inline
bytes::from_array_u8, bytes::to_array_u8 Inline Inline
len(array) Bridge/Exit Inline
get(array) Bridge/Exit Inline for scalar elements, Helper for heap elements
has(array) Bridge/Exit Inline
len(map) Bridge/Exit Inline
get(map) Bridge/Exit Helper
has(map) Bridge/Exit Helper
All other builtins Bridge/Exit Bridge/Exit or NYI, depending on trace shape
Host imports Bridge/Exit Bridge/Exit or branch-exit trace, never inline

Current trace-JIT-wide NYI:

  • traces longer than configured max trace length
  • unsupported opcodes or unsupported trace shapes during recording/lowering
  • unsupported native targets outside x86_64 Windows / Unix-non-macOS and aarch64 Linux / macOS