tclrs 0.2.0

Tcl as a fusevm frontend: a parser and compiler to fusevm::Chunk, with no bespoke VM or JIT
Documentation
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="color-scheme" content="dark light">
  <meta name="description" content="tclrs — Tcl in Rust. A Tcl frontend that parses Tcl source and lowers it to fusevm bytecode, run on the shared language-agnostic bytecode VM. No bespoke VM, no interpreter loop, no code generator. Differentially tested against tclsh 9.0.4. MIT licensed.">
  <title>tclrs — Documentation</title>
  <link rel="preconnect" href="https://fonts.googleapis.com">
  <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
  <link href="https://fonts.googleapis.com/css2?family=Orbitron:wght@400;600;700;900&amp;family=Share+Tech+Mono&amp;display=swap" rel="stylesheet">
  <link rel="stylesheet" href="hud-static.css">
  <link rel="stylesheet" href="tutorial.css">
  <style>
    .tutorial-main { max-width: 68rem; }
    .docs-build-line {
      margin: 0.35rem 0 0;
      font-family: 'Share Tech Mono', ui-monospace, monospace;
      font-size: 11px;
      color: var(--text-dim);
      letter-spacing: 0.03em;
      max-width: 42rem;
      opacity: 0.75;
    }
    .hub-scheme-strip {
      border-bottom: 1px dashed var(--border);
      background: color-mix(in srgb, var(--bg-secondary) 85%, transparent);
      padding: 0.55rem 1.5rem 0.65rem;
      position: relative;
    }
    .hub-scheme-strip-inner {
      max-width: 68rem;
      margin: 0 auto;
      display: flex;
      align-items: center;
      gap: 0.85rem;
    }
    .hub-scheme-strip .hud-scheme-label {
      flex: 0 0 auto;
      font-family: 'Orbitron', sans-serif;
      font-size: 9px;
      font-weight: 700;
      letter-spacing: 2px;
      text-transform: uppercase;
      color: var(--accent);
      text-align: left;
    }
    .hub-scheme-strip .scheme-grid {
      flex: 1 1 auto;
      display: grid;
      grid-template-columns: repeat(5, minmax(0, 1fr));
      gap: 6px;
    }
    @media (max-width: 720px) {
      .hub-scheme-strip-inner { flex-direction: column; align-items: stretch; }
      .hub-scheme-strip .scheme-grid { grid-template-columns: repeat(2, minmax(0, 1fr)); }
    }
    .cat-grid {
      display: grid;
      grid-template-columns: repeat(auto-fill, minmax(16rem, 1fr));
      gap: 0.6rem;
      margin: 0.7rem 0;
    }
    .cat-card {
      border: 1px solid var(--border);
      border-left: 2px solid var(--cyan);
      padding: 0.6rem 0.8rem;
      background: color-mix(in srgb, var(--bg-card) 92%, transparent);
      border-radius: 2px;
    }
    .cat-card h4 {
      font-family: 'Orbitron', sans-serif;
      font-size: 11px;
      font-weight: 700;
      letter-spacing: 1.5px;
      text-transform: uppercase;
      color: var(--cyan);
      margin: 0 0 0.35rem;
    }
    .cat-card p { margin: 0; font-size: 11.5px; color: var(--text-dim); line-height: 1.5; }
    .cat-card code { font-size: 11px; color: var(--accent-light); }
    .reflection-table { width: 100%; border-collapse: collapse; margin: 0.6rem 0 0.2rem; font-size: 12px; }
    .reflection-table th {
      background: var(--bg-secondary); color: var(--cyan);
      font-family: 'Orbitron', sans-serif; font-size: 10px; font-weight: 700;
      letter-spacing: 1px; text-transform: uppercase; text-align: left;
      padding: 6px 10px; border: 1px solid var(--border);
    }
    .reflection-table td { padding: 6px 10px; border: 1px solid var(--border); color: var(--text-dim); vertical-align: top; }
    .reflection-table td code { color: var(--accent-light); background: var(--bg); }
    .status-pill {
      display: inline-block; font-family: 'Orbitron', sans-serif; font-size: 9px;
      font-weight: 700; letter-spacing: 1px; text-transform: uppercase;
      padding: 2px 7px; border-radius: 2px; border: 1px solid var(--border);
    }
    .status-implemented { color: var(--green); border-color: var(--green); }
    .status-progress { color: var(--accent-light); }
    .status-planned { color: var(--text-muted); }
  </style>
</head>
<body>
  <div class="app tutorial-app" id="docsApp">
    <div class="crt-scanline" id="crtH" aria-hidden="true"></div>
    <div class="crt-scanline-v" id="crtV" aria-hidden="true"></div>

    <header class="tutorial-header">
      <div class="tutorial-header-inner">
        <div>
          <h1 class="tutorial-brand">// TCLRS — TCL IN RUST</h1>
          <nav class="tutorial-crumbs" aria-label="Breadcrumb">
            <span class="current">Docs</span>
            <span class="sep">/</span>
            <a href="report.html">Engineering Report</a>
            <span class="sep">/</span>
            <a href="reference.html">Command Reference</a>
            <span class="sep">/</span>
            <a href="https://github.com/MenkeTechnologies/tclrs" target="_blank" rel="noopener noreferrer">GitHub</a>
          </nav>
          <p class="docs-build-line">tclrs v0.1.0 · Tcl on fusevm · parse → bytecode → fusevm VM · no bespoke VM · no interpreter loop · the seventeenth fusevm frontend · MIT</p>
        </div>
        <div class="tutorial-toolbar">
          <button type="button" class="btn btn-secondary" id="btnTheme" title="Toggle light/dark">Theme</button>
          <button type="button" class="btn btn-secondary active" id="btnCrt" title="CRT scanline overlay">CRT</button>
          <button type="button" class="btn btn-secondary active" id="btnNeon" title="Neon border pulse">Neon</button>
          <a class="btn btn-secondary" href="report.html">Report</a>
          <a class="btn btn-secondary" href="reference.html">Reference</a>
          <a class="btn btn-secondary" href="https://github.com/MenkeTechnologies/tclrs" target="_blank" rel="noopener noreferrer">GitHub</a>
        </div>
      </div>
    </header>

    <div class="hub-scheme-strip">
      <div class="hub-scheme-strip-inner">
        <span class="hud-scheme-label">// Color scheme</span>
        <div class="scheme-grid" id="hudSchemeGrid"></div>
      </div>
    </div>

    <main class="tutorial-main">
      <h2 class="tutorial-title"><span class="step-hash">&gt;_</span>TCLRS REFERENCE</h2>
      <p class="tutorial-subtitle">A Tcl frontend written in Rust. A script is parsed once — every substitution the grammar permits resolved at parse time — and each command is lowered to <code>fusevm</code> bytecode, the same bytecode sixteen other language frontends emit. No bespoke VM, no interpreter loop, no code generator in this crate. The reference implementation is <strong>tclsh 9.0.4</strong>, and the test suite compares against it directly. <strong>In active development.</strong></p>

      <section class="tutorial-section">
        <h2>What it is</h2>
        <p><code>tclrs</code> is the seventeenth frontend on <a href="https://github.com/MenkeTechnologies/fusevm">fusevm</a>, the shared bytecode VM behind <code>zshrs</code> (the shell), <code>stryke</code> (the language), <code>awkrs</code> (AWK), <code>vimlrs</code> (VimL), <code>elisprs</code> (Emacs Lisp), <code>rubylang</code> (Ruby), <code>pythonrs</code> (Python), <code>phplang</code> (PHP), <code>node-js</code> (JavaScript), <code>rlang</code> (R), <code>go-rs</code> (Go), <code>arb</code>, and the four JVM frontends. It carries no VM and no code generator of its own: tclsh runs Tcl on its own bytecode engine, tclrs runs Tcl on fusevm's, including fusevm's Cranelift JIT and its ahead-of-time compiler.</p>
        <p>Two properties of Tcl's grammar make ahead-of-time lowering worthwhile. <strong>Braces suppress substitution</strong>, so a braced body is fully known at parse time and compiles once into bytecode instead of being re-parsed on every evaluation — words carry a <code>braced</code> flag for exactly that decision. And <strong>rule 11 rules out rescanning</strong> substituted values, so each character is processed once and the compiler can resolve variable and command references statically wherever the word shape allows.</p>
      </section>

      <section class="tutorial-section">
        <h2>Architecture</h2>
        <p>The pipeline mirrors how <code>zshrs</code> hosts zsh and <code>groovyrs</code> hosts Groovy:</p>
        <pre>Tcl script  →  parser (Script/Command/Word)  →  lower to fusevm bytecode  →  fusevm VM
                                                       │
                                    numeric hook (string operands, integer overflow)
                                    extension ops (floored / and %, integral **, normalize)</pre>
        <div class="cat-grid">
          <div class="cat-card"><h4>fusevm-hosted</h4><p>No local <code>vm.rs</code> / <code>jit.rs</code>. Each command lowers into a <code>fusevm::Chunk</code> and runs on the shared VM (<code>src/compiler.rs</code>).</p></div>
          <div class="cat-card"><h4>No object heap</h4><p>Tcl's value model needs none on top of fusevm's: strings, integers and floats map onto <code>Value</code> directly, and a value keeps its numeric representation until something demands its string form.</p></div>
          <div class="cat-card"><h4>Native arithmetic</h4><p><code>+ - *</code>, the comparisons, the bitwise and shift operators, and short-circuiting <code>&amp;&amp;</code> / <code>||</code> lower to native fusevm ops, so the arithmetic the JIT tiers care about stays visible to them.</p></div>
          <div class="cat-card"><h4>Static stack tracking</h4><p>Each command leaves its result on the stack and the compiler tracks that depth as it goes, so <code>break</code> and <code>continue</code> unwind with a known number of pops rather than a runtime unwinder.</p></div>
        </div>
      </section>

      <section class="tutorial-section">
        <h2>Example</h2>
<pre>set x 5
set y [expr {$x * 2}]
puts "x=$x y=$y"          ;# =&gt; x=5 y=10

puts [expr {-57 / 10}]     ;# =&gt; -6   (Tcl floors toward -inf)
puts [expr {-57 % 10}]     ;# =&gt; 3
puts [expr {2**10}]        ;# =&gt; 1024 (integral ** stays integral)
puts [expr {3.0/2}]        ;# =&gt; 1.5  (Tcl double formatting)
puts [expr {"10" &lt; "9"}]   ;# =&gt; 0    (numeric-preferring comparison)
puts [expr {"abc" lt "abd"}] ;# =&gt; 1  (always-string comparison)

set i 0
while {$i &lt; 3} {
    incr i
    if {$i == 2} { continue }
    puts $i                ;# =&gt; 1, then 3
}</pre>
        <p>Every program in the test suite — this one included — is run through both <code>tclsh</code> and tclrs and the output compared byte for byte. No expected output in the repository is written by hand.</p>
      </section>

      <section class="tutorial-section">
        <h2>Status &amp; roadmap</h2>
        <p>The table below reflects the current state of the tree. See <a href="https://github.com/MenkeTechnologies/tclrs/blob/main/BUGS.md">BUGS.md</a> for the honest ledger of what is not yet carried.</p>
        <table class="reflection-table">
          <thead><tr><th>Component</th><th>State</th><th>Notes</th></tr></thead>
          <tbody>
            <tr><td>Parser — the twelve rules of <code>Tcl(n)</code></td><td><span class="status-pill status-implemented">Implemented</span></td><td>Command/word splitting, quotes, braces, command and variable substitution, the full backslash table, first-word comments, single-pass order (<code>src/parser.rs</code>).</td></tr>
            <tr><td>Script → fusevm bytecode lowering</td><td><span class="status-pill status-implemented">Implemented</span></td><td>Native ops plus frontend extension ops; statically tracked stack depth, backpatched control flow (<code>src/compiler.rs</code>).</td></tr>
            <tr><td>Commands <code>set</code> / <code>puts</code> / <code>expr</code> / <code>incr</code></td><td><span class="status-pill status-implemented">Implemented</span></td><td><code>puts -nonewline</code> supported; command substitution of any implemented command.</td></tr>
            <tr><td>Control flow <code>if</code> / <code>while</code> / <code>foreach</code> / <code>break</code> / <code>continue</code></td><td><span class="status-pill status-implemented">Implemented</span></td><td><code>elseif</code> / <code>else</code> chains; loop exits unwind by a compile-time-known pop count. <code>foreach</code> takes any number of variable lists and value lists, its state carried on the VM stack.</td></tr>
            <tr><td><code>expr</code> — the full <code>expr(n)</code> operator set</td><td><span class="status-pill status-implemented">Implemented</span></td><td>Compiled straight from a braced word with no runtime parse, at <code>expr(n)</code> precedence (<code>src/expr.rs</code>).</td></tr>
            <tr><td>Tcl arithmetic &amp; number formatting</td><td><span class="status-pill status-implemented">Implemented</span></td><td>Floored <code>/</code> and <code>%</code>, integral <code>**</code>, numeric-preferring comparison with string fallback, Tcl double format (<code>src/runtime.rs</code>).</td></tr>
            <tr><td>Differential suites vs <code>tclsh</code></td><td><span class="status-pill status-implemented">Implemented</span></td><td>Parse rules, word splitting, whole-program execution, and generated matrices for list quoting, <code>foreach</code> shapes, glob patterns and index forms, all compared against tclsh 9.0.4; skipped when no tclsh is installed.</td></tr>
            <tr><td>Lists — parsing, canonical quoting, the thirteen list commands, <code>foreach</code>, <code>in</code> / <code>ni</code></td><td><span class="status-pill status-implemented">Implemented</span></td><td>String ⇄ elements ported from <code>TclFindElement</code> and <code>TclScanElement</code> / <code>TclConvertElement</code> (<code>src/list.rs</code>); the commands and <code>lsort</code>'s reference merge sort in <code>src/cmd_list.rs</code>.</td></tr>
            <tr><td>Procedures — <code>proc</code>, <code>return</code>, <code>global</code></td><td><span class="status-pill status-implemented">Implemented</span></td><td>Parameters and locals are frame slots, not globals; signatures collected before emission, so a procedure may call one defined further down (<code>src/procs.rs</code>).</td></tr>
            <tr><td><code>catch</code> / <code>error</code>, <code>for</code>, <code>switch</code></td><td><span class="status-pill status-implemented">Implemented</span></td><td>A <code>catch</code> region records its handler's op index; the driver unwinds the value stack and the call frames to the region's entry state and resumes there (<code>src/control.rs</code>).</td></tr>
            <tr><td>Associative data — array variables, <code>array</code>, <code>dict</code>, <code>unset</code></td><td><span class="status-pill status-implemented">Implemented</span></td><td>Arrays live in the VM's global table keyed by name index (<code>src/assoc.rs</code>).</td></tr>
            <tr><td>Strings — the <code>string</code> ensemble, <code>append</code>, <code>format</code></td><td><span class="status-pill status-implemented">Implemented</span></td><td><code>src/cmd_string.rs</code>. Subcommands and conversions outside the implemented set are refused by name.</td></tr>
            <tr><td>Coroutines — <code>coroutine</code>, <code>yield</code>, <code>yieldto</code>, <code>info coroutine</code></td><td><span class="status-pill status-implemented">Implemented</span></td><td>A coroutine is a second <code>fusevm::VM</code> over the same chunk; the driver owns the transfer and the one global table every context shares (<code>src/coro.rs</code>).</td></tr>
            <tr><td>The <code>tclrs</code> binary, the REPL, and <code>eval</code></td><td><span class="status-pill status-implemented">Implemented</span></td><td>A file, <code>-c script</code>, or stdin, with tclsh's exit statuses and stderr wording. A terminal gets a reedline editor — history, completion drawn from the compiler's own tables, multi-line editing decided by the parser, and procedures that outlive the line defining them; a pipe gets the silent loop. An <code>Interp</code> holds the variables between evaluations, over a source-keyed chunk cache (<code>src/main.rs</code>, <code>src/repl.rs</code>, <code>src/repl_line.rs</code>, <code>src/cache.rs</code>).</td></tr>
            <tr><td>Ahead-of-time compilation — <code>--aot</code></td><td><span class="status-pill status-implemented">Implemented</span></td><td>fusevm's closed-world compiler emits a relocatable object that links against <code>libtclrs.a</code> into a standalone binary. Refused for a script using <code>catch</code> or a coroutine, which need a driver outside <code>VM::run</code> (<code>src/aot.rs</code>).</td></tr>
            <tr><td>JIT tiers armed, and reported — <code>--tiers</code></td><td><span class="status-pill status-progress">Compiles a loop inside a <code>proc</code></span></td><td>Every VM arms fusevm's tracing JIT. A hot <code>while</code> or <code>for</code> loop inside a procedure reaches a compiled trace — <code>traced=true</code>, 3M iterations in 6.6 ms against 243.7 ms interpreted — because its locals are frame slots and every loop is emitted rotated into the do-while shape fusevm's trace compiler accepts. The same loop at a script's top level still reaches nothing: a top-level variable is a VM global, which no tier takes. <code>foreach</code> and <code>dict for</code> reach nothing either — their loop state is frontend extension ops. All measured, not assumed (<code>src/tiers.rs</code>).</td></tr>
            <tr><td><code>{*}</code> argument expansion</td><td><span class="status-pill status-planned">Refused</span></td><td>Recorded on the word by the parser and still refused by the compiler, though the list splitter it needs exists.</td></tr>
            <tr><td>Conformance against the official Tcl test suite</td><td><span class="status-pill status-implemented">Measured</span></td><td>2158 of 5066 attempted cases pass — 42.6%. The suite is fetched and checksum-verified, every case lifted out and run under both interpreters; regenerate with <code>conformance/run.sh</code>. The share is below the previous report's 47.7% because <code>proc</code>, the <code>string</code> ensemble, coroutines and <code>eval</code> landing moved 2,125 cases out of the skip column and into the attempted one: passes went 1404 → 2158, and the denominator went 2941 → 5066 faster.</td></tr>
            <tr><td>Math functions, <code>regexp</code>, channel IO, <code>upvar</code> / <code>uplevel</code>, <code>namespace</code></td><td><span class="status-pill status-planned">Refused</span></td><td>An unknown command is a compile-time error, never a silent miss.</td></tr>
            <tr><td>Arbitrary-precision integers</td><td><span class="status-pill status-planned">Refused</span></td><td>An <code>i64</code> overflow is an error, not a wrap — in the interpreter, under the JIT, and ahead-of-time compiled, because every chunk carries <code>int_overflow_deopt</code>.</td></tr>
            <tr><td>Editor servers — <code>--lsp</code>, <code>--dap</code></td><td><span class="status-pill status-implemented">Implemented</span></td><td><code>--lsp</code> answers diagnostics, completion, hover, signature help and document symbols from the same tables the REPL completes from, deciding what is under the cursor with the module the prompt uses (<code>src/lsp.rs</code>, <code>src/cursor.rs</code>). <code>--dap</code> gives breakpoints, stepping, stack frame and variables by stopping on markers <code>compiler::compile_debug</code> emits and an ordinary compilation does not, so nothing is paid for a debugger that is not attached (<code>src/dap.rs</code>). Both are driven over the wire against the real process by <code>tests/lsp_session.rs</code> and <code>tests/dap_session.rs</code>.</td></tr>
            <tr><td>Inline Rust — <code>rust { ... }</code></td><td><span class="status-pill status-implemented">Implemented</span></td><td>A block is rewritten before parsing into <code>__rust_compile</code>, compiled to a shared library through <code>fusevm::ffi</code> and cached under <code>~/.cache/fusevm/ffi</code> by the SHA-256 of its body; its exports become Tcl commands, registered while the block is <em>lowered</em> rather than when the VM runs (<code>src/rust_ffi.rs</code>).</td></tr>
            <tr><td>The rest of the toolchain — dumps, completion, man pages</td><td><span class="status-pill status-implemented">Implemented</span></td><td><code>--disasm</code>, <code>--dump-tokens</code> and <code>--dump-ast</code> print the bytecode, the lexical output and the parse tree; the zsh completion is <code>completions/_tclrs</code>; the manual pages are <code>tclrs(1)</code> and the all-in-one <code>tclrsall(1)</code>.</td></tr>
          </tbody>
        </table>
      </section>

      <section class="tutorial-section">
        <h2>Why tclrs</h2>
        <div class="cat-grid">
          <div class="cat-card"><h4>Lowered once</h4><p>Braced bodies and braced <code>expr</code> expressions are known at parse time, so they compile once into bytecode instead of being re-parsed per evaluation.</p></div>
          <div class="cat-card"><h4>No string round-trips</h4><p>A value produced as a number stays a number in a VM slot and only acquires a string representation when something asks for one.</p></div>
          <div class="cat-card"><h4>One shared engine</h4><p>VM, JIT and ahead-of-time work in fusevm benefits zshrs, stryke, awkrs, vimlrs, elisprs, rubylang, pythonrs, phplang, node-js, rlang, go-rs, arb, the JVM frontends, and tclrs at once.</p></div>
          <div class="cat-card"><h4>Ported, not reinvented</h4><p>tclsh 9.0.4 is the specification. Expectations are never hand-written: the suites diff tclrs against the reference interpreter directly.</p></div>
        </div>
      </section>

      <section class="tutorial-section">
        <h2>Building from source</h2>
        <p>tclrs builds as a standalone Rust crate (it is not a workspace member of the meta repo):</p>
<pre># clone
git clone https://github.com/MenkeTechnologies/tclrs
cd tclrs

# build
cargo build

# run the suites
cargo test</pre>
        <p>The differential suites invoke <code>tclsh</code> (or <code>tclsh9.0</code> / <code>tclsh8.6</code>) from <code>PATH</code> and report a skip when none is installed, so the suite still runs on a machine without Tcl.</p>
      </section>

      <section class="tutorial-section">
        <h2>Using the library</h2>
<pre>let out = tclrs::eval("set x 5\nputs [expr {$x * 2}]").unwrap();
assert_eq!(out.output, "10\n");</pre>
        <p><code>tclrs::Interp</code> is the same thing with the variables kept between calls — what a REPL needs and what the <code>eval</code> command needs. <code>tclrs::parse</code> returns the parsed <code>Script</code> without running it, for tooling that wants the word structure.</p>
      </section>

      <section class="tutorial-section">
        <h2>License</h2>
        <p>tclrs is <strong>MIT licensed</strong> — free and open source. See <a href="https://github.com/MenkeTechnologies/tclrs/blob/main/LICENSE">LICENSE</a>.</p>
      </section>

      <section class="tutorial-section">
        <h2>Repository &amp; links</h2>
        <ul>
          <li><strong>Engineering report</strong> — <a href="report.html">report.html</a> (architecture, value model, component status, dependency posture)</li>
          <li><strong>Command reference</strong> — <a href="reference.html">reference.html</a> (every command, ensemble subcommand, <code>expr</code> operator and <code>format</code> conversion — generated from the compiler's own tables by <code>cargo run --bin gen-docs</code>)</li>
          <li><strong>Known gaps</strong> — <a href="https://github.com/MenkeTechnologies/tclrs/blob/main/BUGS.md">BUGS.md</a></li>
          <li><strong>Runnable programs</strong> — <a href="https://github.com/MenkeTechnologies/tclrs/tree/main/examples">examples/</a>, one per slice of the language, each checking its own results and gated against tclsh byte for byte</li>
          <li><strong>Source</strong> — <a href="https://github.com/MenkeTechnologies/tclrs">github.com/MenkeTechnologies/tclrs</a></li>
          <li><strong>Issues</strong> — <a href="https://github.com/MenkeTechnologies/tclrs/issues">github.com/MenkeTechnologies/tclrs/issues</a></li>
          <li><strong>The shared VM</strong> — <a href="https://github.com/MenkeTechnologies/fusevm">fusevm</a> (also behind <code>zshrs</code>, <code>stryke</code>, <code>awkrs</code>, <code>vimlrs</code>, <code>elisprs</code>, <code>rubylang</code>, <code>pythonrs</code>, <code>phplang</code>, <code>node-js</code>, <code>rlang</code>, <code>go-rs</code>)</li>
        </ul>
      </section>
    </main>
  </div>

  <script src="hud-theme.js"></script>
</body>
</html>