Crate monty_proto

Expand description

§monty-proto

The wire protocol connecting Monty worker processes to the parents that drive them.

Monty executes untrusted Python, and a Monty process can never be made fully crash-proof against memory errors (stack overflow aborts, allocator aborts — the monty-alloc allocator turns the latter into this crate’s OOM_EXIT_CODE so a parent can classify them). The subprocess architecture isolates those crashes: a parent — the monty-pool crate, and through it the Python and JavaScript packages — drives monty subprocess children over framed stdio (or a WebSocket), and a dead child is simply replaced. This crate defines the protocol both sides speak.

The protocol is protobuf (rather than Monty’s internal CBOR dump format) so a parent or child can be implemented in any language — see proto/monty/v1/monty.proto for the schema and the protocol rules documented alongside it.

§What the crate provides

pb — prost-generated message types. The generated code is checked in; regenerate with make generate-proto (CI enforces sync via make check-proto).
FrameReader / write_frame — 4-byte little-endian length-prefixed framing, with a hard cap on frame length.
Fallible conversions between pb types and Monty’s public types (MontyObject/CallArgs/NamedValues over a MontyGraph, MontyException, mounts, resource limits, …).
Host-object routing on the wire: host-backed ClassInstance / ClassType nodes carry host-generated uuids, and FunctionCall.object_id / NameLookup.object_id route their method calls and lazy attribute lookups back to the parent’s per-session instance store; sandbox-defined classes and instances carry worker-generated uuids that never reach that store. Calling a host-backed instance arrives as a __call__ method call on its uuid.
Host modules: OsCall.import requests a missing top-level module; answer with its value or not_found. Dotted imports read further components as attributes of that value. Before type checking, import_stubs requests host stubs once per module per session. Answer with a dict of dotted module paths to .pyi source or None.
PROTOCOL_VERSION / MIN_SUPPORTED_PROTOCOL_VERSION — the wire schema version a parent declares in Configure, and the range a child serves. Versioned independently of the monty package: peers on different releases interoperate as long as their protocol versions overlap. There is no in-band negotiation, so a child rejecting a version reports its range in the FatalError for the parent to downgrade to, then exits non-zero. Version 0 is always rejected; monty_version is diagnostic metadata, not a compatibility check. Snapshot compatibility is checked separately using the dump-format version.
python (cargo feature, off by default) — the python module: PyO3-based conversions between live Python objects and MontyObject/MontyException, used by the pydantic-monty-client extension module. The feature pulls in pyo3 (but never its extension-module feature — how libpython is linked stays the top crate’s decision), so pure-Rust consumers pay nothing for it.

Repeated fields and byte buffers in protocol messages use BudgetVec<T>. Construct them from standard vectors with .into() or collect an iterator directly; use .into_inner() to recover a standard vector without copying. Host construction and cloning are unbudgeted; fallible try_push charges any growth to the active decode budget.

§Values are special-cased for performance

Values cross as one flat monty.v1.Arena per message: a post-order node arena in which containers hold child indexes. A sub-object shared inside the sandbox, or between two arguments of one call, is sent once, and the carrying message names its roots by index. prost extern_path maps the message onto WireArena, a hand-written prost::Message implementation that encodes borrowed monty_types::unstable::MontyNodes without cloning and decodes one generated protobuf node at a time. Each node is validated and converted before being retained in the domain arena; temporary payloads and conversion allocations share the frame budget. Strings, bytes and reference buffers transfer without copying; reference containers use WireIndexes, WireNodePairs and WireNamedTuple rather than temporary vectors of protobuf ids. Duplicate fields follow protobuf merging rules, and value depth does not increase decoding recursion. tests/differential.rs proves it byte-compatible against a fully prost-generated oracle (tests/oracle/, regenerated and CI-checked together with the main codegen).

§Children are untrusted

A parent must treat every frame from a (possibly compromised) child as untrusted input: conversions from proto to Rust are fallible by design, decoding enforces a per-frame decode budget and validates every arena index, and nothing in this crate panics on malformed wire data.

Decode protocol types through decode_frame or FrameReader::read. These functions manage the per-frame allocation budget automatically. Direct Message::decode calls on these types fail if they allocate payload storage without a frame budget.

Frames are capped at 256 MiB, with a separate fixed 1 GiB budget for cumulative decoded allocation requests. This budget is independent of the session’s max_memory. Compact messages can require much more memory when decoded. The budget covers arena slots, repeated-field capacity, strings, byte buffers, boxed payloads and BigInt storage. Temporary protobuf buffers and conversions into domain nodes share this budget. Child references include an allowance for host container storage; shared sub-objects are encoded once. Growth charges the full replacement allocation, with no refunds for discarded payloads. A frame can therefore exceed the budget even if its final decoded value occupies less than 1 GiB. The receiver checks the budget before allocating payload storage.

The wire buffer, bounded stack and error storage, allocator metadata and subsequent host conversions are not counted. Each concurrent decode has its own budget; this is not a process-memory limit. See DEFAULT_MAX_DECODE_BYTES in src/frame.rs for the accounting contract. The browser component uses separate decoded-value estimates for WIT arenas, with the same 1 GiB ceiling. Those checks do not account for all allocations made by the component ABI or JavaScript conversion.

Invalid dates, timedeltas, exception names and other semantic values are rejected after parsing each protobuf node. The browser component validates semantic values while converting WIT arenas. A parent receiving an invalid frame discards the worker with a protocol error. A worker receiving such a malformed request reports RuntimeError("protocol violation: malformed request: ...") and keeps the session.

§Worker state machine

The worker cargo feature (off by default) adds the worker module: the transport-agnostic child state machine, shared by the native monty subprocess worker and the wasm worker. It links the monty interpreter, so only worker-side crates enable it.

An external FunctionCall with allow_eager_await = true permits the parent to await a coroutine before replying. The parent sends its value or exception in ResumeFutures, with exactly one result matching the call ID. The worker creates a settled awaitable and continues, avoiding a separate ResolveFutures suspension. Synchronous returns still use ResumeCall; parents may also ignore the hint and register a pending future as before. Older workers omit the flag, which defaults to false, so newer parents retain the existing reply sequence.

§Monty crates

monty — the core interpreter: Python parser, bytecode VM, and sandbox.
monty-types — the shared boundary data types (values, exceptions, OS calls, resource limits) hosts use without linking the interpreter.
monty-fs — host-side filesystem mounts: maps virtual sandbox paths to real host directories.
monty-runtime — the monty binary: REPL, file runner, and subprocess worker mode.
monty-pool — an elastic pool of crash-isolated monty worker subprocesses.
monty-proto — the protobuf wire protocol spoken between pool parents and workers. this crate
monty-type-checking — type checking of sandboxed code, powered by ty.
monty-typeshed — the trimmed typeshed stubs describing the stdlib subset Monty implements.
monty-macros — the proc macros behind monty’s argument parsing.

Modules§

pb

Structs§

BudgetVec: A protocol vector whose decoding growth is charged before allocation. Conversion to and from a standard vector transfers ownership without copying. Host construction and cloning do not require an active decode budget. In-place growth must propagate a decode error:
FrameReader: Reads length-prefixed protobuf frames from a byte stream.
WireArena: The wire form of a MontyGraph: what the monty.v1.Arena proto message decodes into and encodes from.
WireFunctionCall: Wire form of monty.v1.FunctionCall: the call’s argument arena plus the ids of its positional and keyword arguments.
WireIndexes: Wire Indexes stored as domain ids, avoiding an intermediate Vec<u32>.
WireNamedTuple: Wire named tuple with domain ids and budgeted names, ready for zero-copy conversion.
WireNodePairs: Wire NodePairs stored as domain pairs, with no intermediate pair vector.

Enums§

FrameError: Framing or decoding failure while reading or writing protocol messages.
ProtoConvertError: Why a wire value could not be converted into its monty equivalent.

Constants§

DEFAULT_MAX_DECODE_BYTES: Per-frame ceiling on cumulative decoded allocation requests (1 GiB). Generated protocol messages and hand-written values budget vectors, buffers and boxes before allocating, including full replacement buffers on growth. Freed payloads are not refunded. Excludes the wire buffer, bounded decode stack/error overhead and allocator metadata; multiplies per concurrent decode.
DEFAULT_PRINT_FLUSH_INTERVAL: How long the child holds buffered print() output before emitting it as a Print event, when pb::Configure::print_flush_interval_ms says nothing.
MAX_FRAME_LEN: Default maximum frame length (256 MiB).
MIN_SUPPORTED_PROTOCOL_VERSION: Oldest PROTOCOL_VERSION this build still serves.
PROTOCOL_VERSION: Version of the wire schema this build speaks, sent in pb::Configure::protocol_version and range-checked by the child.

Functions§

check_protocol_version: Checks a peer’s declared pb::Configure::protocol_version against the range this build serves.
decode_frame: Decodes one already-deframed message from bytes.
encode_framed_into: Encodes msg as one length-prefixed frame — prefix and body in a single buffer — into buf (cleared first), enforcing MAX_FRAME_LEN before encoding.
encode_to_capped_vec: Encodes msg to a Vec<u8>, enforcing MAX_FRAME_LEN before encoding (so a >256 MiB message is rejected without allocating it).
exceeds_max_frame_len: The encoded frame length of msg when it exceeds MAX_FRAME_LEN (saturating at u32::MAX), or None when it fits.
ext_result_from_proto: Validates a decoded call result against the arena its message carried.
ext_result_to_proto: Splits a call result into its wire kind and the arena a Return value indexes (None for the other arms).
future_results_from_proto: Converts wire future results into (call_id, result) pairs for ResolveFutures::resume; each returned value gets its own copy of the nodes it reaches.
future_results_to_proto: Merges every returned value into one arena and points each ReturnValue into it.
named_values_from_proto: Validates decoded named inputs against their arena.
named_values_to_proto: Splits named inputs into NamedRefs and the arena they index.
os_call_from_proto: Validates a decoded OS call envelope into its call id and typed call.
os_call_to_proto: Builds the OsCall envelope: call id, typed arm, the eager-await hint, the call’s source position and, for Getenv, the arena its default indexes.
resume_call_from_proto: The result of a ResumeCall reply, read with its arena by ext_result_from_proto.
validate_requirement: Rejects a requirement string that uv would interpret as a command-line option rather than a package specifier.
write_frame: Encodes msg and writes it to writer as one length-prefixed frame, then flushes (see the module docs for why flushing every frame is required).