pythonrs 0.1.2

Python as a fusevm frontend: lexer, parser, and compiler to fusevm::Chunk on a PyHost heap — the first compiled standalone Python runtime
Documentation
██████╗ ██╗   ██╗████████╗██╗  ██╗ ██████╗ ███╗   ██╗██████╗ ███████╗
██╔══██╗╚██╗ ██╔╝╚══██╔══╝██║  ██║██╔═══██╗████╗  ██║██╔══██╗██╔════╝
██████╔╝ ╚████╔╝    ██║   ███████║██║   ██║██╔██╗ ██║██████╔╝███████╗
██╔═══╝   ╚██╔╝     ██║   ██╔══██║██║   ██║██║╚██╗██║██╔══██╗╚════██║
██║        ██║      ██║   ██║  ██║╚██████╔╝██║ ╚████║██║  ██║███████║
╚═╝        ╚═╝      ╚═╝   ╚═╝  ╚═╝ ╚═════╝ ╚═╝  ╚═══╝╚═╝  ╚═╝╚══════╝

CI Rust Docs license status

[PYTHON, COMPILED TO BYTECODE — rkyv-CACHED ON EVERY RUN, AOT-NATIVE]

"CPython compiles to its own bytecode and walks it. pythonrs lowers Python to a shared machine, caches the result on every run, and can bake a script into a native binary."

pythonrs is Python as a fusevm frontend — a lexer/parser and compiler that lowers Python 3 to fusevm::Chunk bytecode running on the fusevm three-tier Cranelift JIT, over a PyHost object heap. There is no bespoke VM and no bespoke JIT: pythonrs is a pure front end; execution and codegen live in fusevm — the same engine behind zshrs, strykelang, awkrs, vimlrs, elisprs, and rubylang.

It is the first compiled standalone Python runtime that both transparently caches bytecode via rkyv on every run and AOT-compiles a script to a native executable.

Read the Docs · Engineering Report · Builtin Reference


Table of Contents


[0x00] OVERVIEW

pythonrs keeps Python the language and throws away CPython's execution model. It lexes and parses Python to an AST, lowers the AST to fusevm bytecode, and runs the bytecode on a stack VM with a Cranelift JIT. Arithmetic and comparisons lower to native ops; Python-specific behavior — truthiness, str/list concat, bignum promotion, attribute and method dispatch — runs through a strict numeric hook and a numbered builtin-call protocol into the PyHost object heap.

Two things set it apart from every other standalone Python:

  • Transparent rkyv bytecode cache — on every run. python foo.py hashes the source, consults ~/.pythonrs/scripts.rkyv, and on a hit runs the compiled chunks directly with lex/parse/lower skipped entirely. No flags, no separate build step, no __pycache__ ritual.
  • AOT to a native executable. python --build foo.py emits a standalone native binary (via fusevm::aot, linked against the pythonrs runtime staticlib) that runs the script with no interpreter present.

[0x01] INSTALL

# Via the Homebrew tap (bumped by each release; formula is `pythonrs`)
brew install menketechnologies/menketech/pythonrs

# Or from source
git clone https://github.com/MenkeTechnologies/pythonrs
cd pythonrs && cargo build --release
# binary: target/release/python  (+ libpythonrs.a for AOT linking)

Zsh tab completion

cp completions/_python "${fpath[1]}/"
# or: fpath=(/path/to/pythonrs/completions $fpath) in .zshrc

[0x02] USAGE

python foo.py               # run a script (transparently rkyv-cached)
python -c 'print(1 + 1)'    # run a one-liner
python --build foo.py       # AOT-compile to a native ./foo executable
python --dump-bytecode f.py # print the lowered fusevm bytecode
python --tiers f.py         # run it, then report which fusevm tiers took it
python --repl               # interactive REPL
python --lsp                # Language Server Protocol over stdio
python --doctor             # runtime / CPython / cache / env diagnostic report
python --cacheview          # list the compiled programs in the bytecode cache
python --cache-clear        # delete the bytecode cache shard

The REPL is a reedline line editor: Tab pops a columnar completion menu (Shift+Tab / BackTab cycles backward). On a bare word it offers the language keywords, builtins, math.*, per-type method names, and the live module globals / class names of the persistent session. After a name. it switches to type-aware attribute completion — it reads the receiver's live runtime type and offers exactly that surface: str/list/dict/set/tuple/int/float methods for a builtin value, an imported module's own namespace, or an instance's attributes plus every method reachable along its class MRO. History persists to ~/.pythonrs/history.

Set PYTHONRS_TRACE=1 to log cache hit/miss to stderr (silent otherwise). Set PYTHONRS_CACHE=0 (or false/no) to disable the transparent bytecode cache entirely — every run recompiles and nothing is stored.

[0x03] LANGUAGE FEATURES

Arbitrary-precision integers, real closures, classes with inheritance, operator dunders, generators (yield / yield from / lazy generator expressions, backed by stackful corosensei coroutines), match/case structural pattern matching, own-scope comprehensions (list/dict/set) and proper nonlocal, f-strings, exceptions, and full call-site and literal * / ** unpacking. The PyHost heap implements the str / list / dict / tuple / set / instance object model with the operator, attribute, item, and iteration protocols. See [0x07] and BUGS.md for the honest list of what is not yet implemented.

[0x04] COMMAND-LINE FLAGS

Flag Effect
(none) Run the script/one-liner, transparently rkyv-cached.
-c SRC Execute a one-liner (python -c 'print(1+1)').
-m MODULE … Run a library module as a script. Delegates to the embedded CPython (runpy), so -m pip / -m venv / -m http.server / -m json.tool behave exactly like python3 -m; every token after the module is the module's own sys.argv. Needs the stdlib-ffi bridge (default build).
-u Force unbuffered stdout/stderr (PYTHONUNBUFFERED).
-E -I -O -S -B -W CPython interpreter flags, accepted for drop-in compatibility (-u/-W take real effect via the embedded interpreter; the rest are tolerated no-ops).
--build AOT-compile the script to a standalone native executable.
--dump-bytecode Print the lowered fusevm bytecode and exit.
--tiers Run the script, then report which fusevm execution tier took each of its chunks.
--repl Start the interactive REPL.
--lsp Run the Language Server Protocol server over stdio.
--doctor Print a diagnostic report — runtime, embedded CPython, fusevm engine, bytecode cache, PYTHON* env, and every python* interpreter on PATH — and exit.
--cacheview List the compiled programs held in the rkyv bytecode cache (~/.pythonrs/scripts.rkyv): per-entry hashes, blob size, and op/function/try/warning counts.
--cache-clear Delete the rkyv bytecode cache shard and exit.

[0x05] ARCHITECTURE

lexer  →  parser  →  AST  →  compiler  →  fusevm::Chunk  →  fusevm VM + JIT
                                              │                    │
                                              └── CallBuiltin ─────→ host (PyHost heap)
  • lexer.rs — indentation-significant tokenizer (INDENT/DEDENT/NEWLINE, f-strings).
  • parser.rs — recursive-descent Python grammar → ast.rs.
  • compiler.rs — lowers the AST to fusevm ops + CallBuiltin dispatches.
  • host.rs — the PyHost object heap (str/list/dict/tuple/set/instances/…), the operator/attribute/item/iteration semantics, and the fusevm run plumbing.
  • builtins.rs — the CallBuiltin handler table, the numeric hook, the Kernel builtin functions (print / len / range / …), and per-type methods.
  • cache.rs — the rkyv-shard bytecode cache.
  • aot_native.rs — native-executable emission via fusevm::aot.
  • pylib/ — the vendored CPython pure-Python standard library (.py sources) shipped with pythonrs. In the native build these are imported by compiling and executing them on pythonrs's own interpreter — no libpython.

Standard library: two build modes

The import path resolves a module from native inline arms first, then:

Build Command import <stdlib> source
Native (CPython-free) cargo build --no-default-features The vendored pylib/*.py, compiled on pythonrs and run on fusevm. No pyo3, no libpython — CPython is not in the dependency graph. This is the shipping target (brew install pythonrs lays pylib/ beside the binary).
Bridged (drop-in) cargo build The real CPython stdlib over an embedded libpython (pyo3, the stdlib-ffi feature). Kept primary while the native build's C-accelerator floor (posix/_io/_sre/…) is completed.

Imports are memoized through the host's sys.modules cache, so a module's vendored .py executes at most once (CPython run-once identity semantics). $PYTHONRS_LIB overrides the pylib/ search path.

[0x06] PARITY HARNESS

Correctness is measured, not asserted: an example corpus runs through both pythonrs and the reference python3, and the output is diffed byte-for-byte. pythonrs runs a large, real subset of Python 3, verified against CPython on that corpus.

Beyond the fixed corpus, the parity-fuzz binary is a differential fuzzer. It generates thousands of grammar-driven, deterministic-output snippets — biased toward the historically fragile areas (float repr, integer ///% sign rules, bignum, slices, the format mini-language, string methods) — runs each through python -c and the reference python3 -c, and reports every case where stdout or accept/reject diverges. Each case is seeded, so any divergence is delta-debugged to a minimal reproducer and replays exactly:

cargo build --bin parity-fuzz
./target/debug/parity-fuzz --count 5000          # fuzz every mode
./target/debug/parity-fuzz --formatspec          # one surface only
./target/debug/parity-fuzz --seed 51 --once      # replay + minimize one case

The generator never emits nondeterministic output, so every reported divergence is a real gap. PYTHONRS_FUZZ_PYTHON names the reference interpreter; a --baseline allowlist keeps known gaps from failing while new ones exit non-zero.

[0x07] STATUS & ROADMAP

Active, in development. The runtime executes a substantial real subset of Python 3. The full CPython standard library is importable by default — the stdlib-ffi bridge (on by default) delegates import os/re/json/… to an embedded libpython, so only a --no-default-features build is limited to the native module subset. BUGS.md is the honest ledger of remaining gaps. A DAP debug adapter (--dap) — source-line and function breakpoints, stepping, call stack, locals, and expression evaluate — ships today, alongside man pages and the generated reference.html.

[0x08] DOCUMENTATION

[0xFF] LICENSE

MIT.