# monty-pool
A pool of [Monty](https://github.com/pydantic/monty) worker processes for running untrusted
Python code with crash isolation.
Monty executes untrusted Python, and a Monty process can never be made fully crash-proof
against memory errors (stack overflow aborts, allocator aborts). This crate isolates those
crashes by running the interpreter **only in worker subprocesses**, reached over Monty's wire
protocol: a crashed worker kills only itself, the pool detects the death and replaces the
worker, and the parent process is never at risk.
This is the recommended way to run Monty from Rust. It is also the engine underneath the
[`pydantic-monty`](https://pypi.org/project/pydantic-monty/) Python package and the
[`@pydantic/monty`](https://www.npmjs.com/package/@pydantic/monty) JavaScript package.
## Model
A `Pool` keeps an elastic set of workers (`min_processes` prewarmed, up to `max_processes`).
`Pool::checkout` dedicates one worker to one REPL session: the caller feeds snippets of code
and answers suspension events (`TurnEvent` — external function calls, OS calls, name lookups,
async futures) until the snippet completes, then `Checkout::finish` returns the worker to the
pool for reuse. A `Checkout` dropped without `finish` kills its worker instead — mid-execution
state cannot be trusted back into the pool.
The pool is async end-to-end and runs on [tokio](https://tokio.rs): frame reads are
cancel-safe (partial-frame state lives in the worker), and turn deadlines are ordinary timers
rather than a watchdog thread. Turn futures are not resumable after being dropped mid-flight —
the checkout notices, discards the worker, and fails the next call cleanly.
## Usage
Workers are `monty` CLI binaries spawned as subprocesses — build one with
`cargo build -p monty-runtime` in the [Monty repository](https://github.com/pydantic/monty), or
install it from PyPI as [`pydantic-monty-runtime`](https://pypi.org/project/pydantic-monty-runtime/).
```rust,no_run
use std::time::Duration;
use monty_pool::{Pool, PoolConfig, PoolError, ReplConfig, TurnEvent, on_print_sync};
#[tokio::main]
async fn main() -> Result<(), PoolError> {
let mut config = PoolConfig::subprocess("path/to/monty");
// no timeouts by default; set one before running untrusted code
config.request_timeout = Some(Duration::from_secs(30));
let pool = Pool::new(config).await?;
let mut session = pool.checkout(&ReplConfig::default()).await?;
let mut on_print = on_print_sync(|_stream, text| print!("{text}"));
// session state persists between feeds on the same checkout
session.feed("x = 21", vec![], vec![], false, &mut on_print).await?;
let event = session.feed("x * 2", vec![], vec![], false, &mut on_print).await?;
match event {
TurnEvent::Complete(value) => println!("result: {value}"), // 42
// other events are suspensions (external function calls, OS calls,
// name lookups, futures) answered with `resume` / `resume_name_lookup`
// / `resume_futures` to continue the turn
other => println!("suspended: {other:?}"),
}
// return the worker to the pool for reuse by the next checkout
session.finish().await?;
Ok(())
}
```
`ReplConfig` sets per-session `ResourceLimits`, type checking of every snippet, and `print_flush_interval`.
The flush interval batches `print()` output; `Duration::ZERO` sends one event per completed line.
Its `os_policy` sets the clock, timezone, initial random state and sleep policy for the session.
`CallHost` delegates calls to the caller's OS handler through `TurnEvent::OsCall`.
Every suspension variant of `TurnEvent` carries `position`, a `SourceRange` locating the suspending expression.
The default `SleepMode::System` sets `system_sleep` to the capped delay for the caller to await directly.
`SleepMode::Zero` returns immediately.
`Checkout::feed` accepts inputs exposed as sandbox globals and per-feed filesystem mounts (`MountSpec`); mounts that
overlap on the host or repeat a virtual path fail the feed with a session-preserving `PoolError::Runtime`.
`Checkout::feed_with_cwd` also changes the working directory, which defaults to the first feed's first mount and persists.
`Checkout::worker_id` identifies a worker within its pool independently of PID reuse, for either transport.
It returns `None` after the worker is released or discarded; `Checkout::pid` remains the subprocess-only OS diagnostic.
`Checkout::dump` snapshots a session; `Checkout::restore` can restore it on another worker or machine.
The caller must establish that restored bytes are unmodified output from a trusted, compatible Monty producer.
Neither the pool nor the interpreter authenticates snapshots; successful loading does not establish validity.
Invalid snapshots have no correctness or availability guarantees.
## Protections over in-process execution
- **Crash isolation** — a segfault, stack-overflow abort, or allocator abort in the sandbox
kills only the worker. The pool observes the death as `PoolError::Crashed`, discards the
worker, and spawns a replacement; the parent process and every other session stay healthy.
- **Hard timeouts** — a parent-side deadline kills any worker whose turn exceeds
`request_timeout` (`PoolError::Timeout`), backstopping the sandbox's own resource limits
and catching hangs those limits cannot see. Synchronous host telemetry processors delay
enforcement while they run because the timer cannot be polled. When a session has a
`max_feed_duration` or `max_turn_duration` budget, the deadline also enforces it from outside the child,
each with its own grace (`feed_duration_limit_grace`, `turn_duration_limit_grace`, 1s by default;
`None` disables that backstop).
A `max_suspensions` budget is enforced by the pool alone: it counts the suspensions it services
and ends the feed past the budget with an uncatchable `RuntimeError` in the sandbox.
`PoolConfig::subprocess` sets neither `request_timeout` nor `checkout_timeout` by
default; set `request_timeout` yourself for untrusted code.
- **Untrusted children** — the parent treats every frame from a (possibly compromised)
worker as untrusted: wire decoding validates everything and never panics, and a worker
that violates the protocol is discarded.
- **Worker recycling** — `max_checkouts_per_worker` recycles long-lived children to bound
the impact of any slow leak.
- **Memory limits** — a session's `max_memory` also caps the worker's live allocations,
enforced in the worker's own global allocator
([`monty-alloc`](https://crates.io/crates/monty-alloc)) plus 4 MB of headroom (32 MB with
type checking), rather than letting a worker grow the host until the OOM killer
intervenes. Exceeding it, or a refused allocation, exits the worker with a dedicated code
so it is reported as `PoolError::Runtime`/`MemoryError` instead of an unclassifiable abort.
The worker is already dead when the host receives this error.
Ordinary sandbox exceptions leave the session usable.
After a soft memory or time limit, the worker survives but the heap has no correctness guarantees.
Later feeds may still succeed: the duration budgets restart at the next feed, and a soft memory limit does not end
the session either.
Discard it yourself.
A failed restore also discards the worker.
Timeouts kill the single worker PID, not a process group; the Monty sandbox must never spawn subprocesses.
A remote CPython worker needs deployment-level process teardown instead.
The duration backstop trusts the worker's reported execution time: under-reporting can stretch each turn to the
full budget plus grace, but `request_timeout` applies independently.
Both deadlines are polled, so decoding a large reply can delay enforcement.
Host mount I/O runs between turns and is not covered by either deadline; see
[filesystem timeouts](https://github.com/pydantic/monty/blob/main/docs/filesystem.md#io-timeouts-and-cancellation).
## Observability
The optional `telemetry` feature records semantic execution for language bindings and other hosts.
The former `telemetry-adapter` feature and `telemetry_adapter` module remain compatibility aliases.
`monty-pool` never selects or shuts down a network exporter or reads exporter credentials; the host SDK owns those
choices and its final flush/shutdown.
Recording happens in the host process, which builds every request and decodes every event
anyway, so both transports are covered and the workers stay uninstrumented. Each instrumented checkout
becomes one session span; each feed is a nested span held across suspension round-trips, with
a child span per suspension whose duration is the host round-trip. Fed code, inputs, call
arguments and results, exceptions and `print` output are recorded in full — values encoded
the way the Python logfire SDK encodes attributes, capped at 64KB per value — while
`Load`/`Dump` snapshot blobs are recorded by size only. Supplying an SDK is therefore an
explicit opt-in to recording potentially sensitive values.
The adapter configures a process-global Rust pipeline and returns a handle that creates each
checkout's serialized parent context. Span and log records are emitted through
`TelemetryAdapter`. A binding can either receive standard OTLP protobuf batches aggregated by
the Rust pipeline or receive each raw measurement for aggregation by its native SDK. Python,
Node, and third-party bindings retain ownership of their native SDK and exporter. Without the
feature, workers contain no telemetry recorder or telemetry hot path.
### Metrics
The same handle yields a `Metrics` for `PoolConfig::metrics` — as does
`Metrics::for_logfire` for a Rust host. `Metrics::for_logfire` records directly into the Rust
host's configured Logfire instruments. A language adapter either records into the statically
linked SDK or streams raw measurements to the foreign host. Either turns on the aggregate side:
pool health
(`monty.pool.workers.live`, `monty.pool.workers.idle`,
`monty.pool.workers.suspended`, `monty.pool.checkout.wait`, `monty.pool.worker.terminated`,
`monty.pool.session.duration`, `monty.pool.session.resumed`) and per-turn cost (`monty.run.duration`,
`monty.run.execution_time`, `monty.turn.duration`, `monty.run.suspensions`,
`monty.ext.call.duration`, `monty.snapshot.bytes`, `monty.print.bytes`,
`monty.wire.frame.bytes`).
`monty.pool.session.duration` uses `ok` for a clean finish, `error` when the worker is lost, and `abandoned` when a
live checkout is dropped.
`monty.pool.session.resumed` counts auto-resume attempts: `ok`, or why the reload failed (`exhausted`,
`disconnected`, `refused`, `shutdown`, `mismatch`, `timeout`).
Two differences from the spans above. Metrics cover **every** checkout, not only the ones a
host gave a parent context — an aggregate over traced sessions alone would be misleading —
and they record no sandbox-supplied values at all: every attribute is a closed set, so a
called function's name (under any outcome — a host lookup that is a callable resolves
anything), an exception class and any path are all left out rather than becoming a time
series each. The one name recorded is an os call's, which comes from the protocol's own
fixed set. The subtraction worth knowing: `monty.run.duration` minus
`monty.run.execution_time` is host and transport overhead, primarily time spent answering
suspensions.
Metric attributes deliberately never identify a pool. The worker up/down counters therefore
total over all pools recording into the same host meter, and a dropped pool subtracts its
remaining contribution.
`configure_telemetry_adapter` delivers aggregated `ExportMetricsServiceRequest` protobufs
through `TelemetryAdapter::export_metrics`; its flush path should call
`TelemetryAdapterHandle::force_flush`. `configure_telemetry_adapter_with_host_metrics` instead
delivers every `Measurement` through `TelemetryAdapter::record_metric`, allowing the foreign
SDK's views, readers, temporality, and exporters to apply. The adapter method is called
synchronously, but a language bridge may queue the measurement before invoking its SDK and must
drain that queue before the SDK is flushed or shut down. Both callbacks default to dropping their
input so adapters that do not support metrics continue to work.
## Transports
- **Subprocess** (`PoolConfig::subprocess`) — spawn local `monty subprocess` children over
framed stdio. These are the poolable workers: prewarmed, reused across checkouts, and
replaced on crash.
- **WebSocket** (`PoolConfig::websocket`) — dial a remote child (or a relay pairing the two
ends) over `ws://`/`wss://`. These workers are single-use: dialed fresh per checkout,
never prewarmed or returned to the pool. Isolation is the remote host's responsibility —
a remote crash is observed as the connection dropping. `Pool::checkout_with` takes
`CheckoutOptions::connect_headers`, extra headers for that checkout's upgrade request —
e.g. a token for a relay in front of the worker. The request carries
`User-Agent: monty-pool/<version>`, and with the `telemetry` feature the `traceparent`
(and `tracestate`) of `CheckoutOptions::telemetry`, so server-side spans join the
caller's trace; a `connect_headers` entry of the same name replaces either.
A WebSocket connection lost mid-session reports `PoolError::Disconnected`; it cannot distinguish a worker crash
from a server policy drop.
A draining server can instead return `PoolError::Shutdown`, naming what to load the session from when it could store
it.
The interrupted request did not run, but restoring a suspended dump repeats its host call, which may already have
had side effects; callbacks used this way should be idempotent.
A local subprocess claiming shutdown is a protocol violation.
A remote that supports persistence names sessions with an opaque ID, `Checkout::session_id`;
`ReplConfig::persistence` asks it to store the session or not, and subprocess workers ignore both.
`ReplConfig::profile` names a server-side profile to run the session under; it is sent in `Configure` as-is, and
subprocess workers ignore it too.
`Checkout::restore` can accept this ID instead of dump bytes to restore a session's state after a disconnect, whether
intentional or due to parking from e.g. an idle timeout or a remote restart.
The remote is free to determine what `Checkout::restore` will do, for example it may lock the existing session to
other consumers or it may issue a new session.
A remote without persistence refuses `Checkout::dump` and `Checkout::restore` with `PoolError::Runtime`, and the
session carries on.
With `PoolConfig::auto_resume` (the default), a shutdown answering a named session's request is not returned: the
checkout redials, loads what the `ShutdownDump` named into a new session, re-sends the request and adopts the new
session's ID.
The session's host-counted suspension and sleep totals carry over, so a shutdown grants no extra allowance.
It returns the original `PoolError::Shutdown` if the shutdown named nothing to load, if the new connection fails, or
if the new session is not in the state the old one was in at the shutdown.
The redial reuses the checkout's original upgrade headers, so a short-lived token in them can make the resume fail.
A bare disconnect is never resumed, since the request may have run.
Before type checking a feed, `import_stubs` requests host module stubs once per module per session.
Answer with a dict of dotted module paths to `.pyi` source or `None`, or `NotHandled` for no stubs.
## Monty crates
- [`monty`](https://crates.io/crates/monty) — the core interpreter: Python parser, bytecode VM, and sandbox.
- [`monty-types`](https://crates.io/crates/monty-types) — the shared boundary data types (values, exceptions, OS calls, resource limits) hosts use without linking the interpreter.
- [`monty-fs`](https://crates.io/crates/monty-fs) — host-side filesystem mounts: maps virtual sandbox paths to real host directories.
- [`monty-runtime`](https://crates.io/crates/monty-runtime) — the `monty` binary: REPL, file runner, and subprocess worker mode.
- [`monty-pool`](https://crates.io/crates/monty-pool) — an elastic pool of crash-isolated `monty` worker subprocesses. **this crate**
- [`monty-proto`](https://crates.io/crates/monty-proto) — the protobuf wire protocol spoken between pool parents and workers.
- [`monty-type-checking`](https://crates.io/crates/monty-type-checking) — type checking of sandboxed code, powered by [ty](https://docs.astral.sh/ty/).
- [`monty-typeshed`](https://crates.io/crates/monty-typeshed) — the trimmed typeshed stubs describing the stdlib subset Monty implements.
- [`monty-macros`](https://crates.io/crates/monty-macros) — the proc macros behind `monty`'s argument parsing.