# Reactor, I/O, and Process Spawning
CoreShift-Core implements a high-performance, non-blocking I/O system based on the Linux `epoll` facility, plus the process-spawning lifecycle that consumes it.
## Design Philosophy
The I/O subsystem is designed to be **explicit** and **stateless**. It does not include a background thread pool or a complex async runtime. Instead, it provides the building blocks for callers to drive their own event loops.
## Components
### `Reactor`
The `Reactor` is a thin wrapper around a Linux `epoll` file descriptor.
- **Edge-Triggered**: Core uses `EPOLLET` (edge-triggered) readiness. This means that when a readiness event occurs, the caller **must** continue reading or writing to the descriptor until it receives an `EAGAIN` or `EWOULDBLOCK` error.
- **Tokens**: Every registered descriptor is associated with a `Token` (an opaque `u64`). This allows the caller to quickly identify which component is ready for I/O.
- `Fd::dup` (in `fd`) fans an owned descriptor out to multiple threads.
Full `Reactor` surface:
- `new()` — `epoll_create1(EPOLL_CLOEXEC)`.
- `add(&Fd, readable, writable)` — register, **always edge-triggered**, returns
a fresh `Token` (tokens start at 1, monotonically increasing, never reused).
- `add_priority(&Fd)` — register for `EPOLLPRI` only (edge-triggered).
- `add_with_flags(&Fd, flags)` — raw `epoll_ctl(EPOLL_CTL_ADD)`; flags pass
through verbatim (level-triggered / `EPOLLONESHOT` possible here).
- `del(&Fd)` — unregister. There is **no modify/`EPOLL_CTL_MOD`** API;
re-registration requires `del` + `add`.
- `setup_signalfd()` — blocks `SIGCHLD` on the calling thread, creates a
`signalfd` (non-blocking + `CLOEXEC`), registers it readable; returns its
`Token`. Double init → `EINVAL`. The prior mask is restored on `Drop` only if
drop runs on the same thread.
- `setup_inotify()` — creates an inotify fd (`IN_NONBLOCK|IN_CLOEXEC`),
registers it readable; returns `(Fd, Token)`.
- `drain_signalfd()` — loop-reads signalfd records until `EAGAIN` (no-op if no
signalfd is set up).
- `wait(&mut events, max, timeout)` — fills `events` with the fresh batch
(the buffer is cleared first); `timeout` is `-1` indefinite / `0` poll /
`>0` ms. `EINTR` maps to `Ok(0)` (treated as a timeout). Event mapping:
`readable = EPOLLIN|EPOLLERR`, `writable = EPOLLOUT|EPOLLERR`,
`priority = EPOLLPRI|EPOLLERR`, `error = EPOLLERR`, `hangup = EPOLLHUP`.
- The `Reactor` does not own fds and never dups — the caller must keep its
`Fd` alive (closing it auto-removes it from epoll per kernel semantics).
### `Fd`
`fd::Fd` is an owned file descriptor wrapper. It ensures that descriptors are closed correctly on drop and provides safe helper methods for `read`, `write`, `eventfd`/`timerfd` creation, `dup`/`dup2`, non-blocking flags, and u64 read/write. `Token`/`Event` live here too (shared by `reactor`).
### `DrainState`
`DrainState` is a high-level orchestrator for process I/O. It manages the bookkeeping for a process's `stdin`, `stdout`, and `stderr` pipes.
- **Multiplexing**: It coordinates reading from multiple output streams into internal buffers while respecting a global memory limit.
- **Bidirectional**: `write_stdin` pushes bytes to the child's stdin; `read_fd` drains an output stream.
- **Pause/Resume**: `resume_stdout` / `resume_stderr` restart a paused stream. Pausing is driven **only** by streaming backpressure — a `ChunkSink` returning `SinkResult::Pause` (sink queue full). The output limit never pauses; exceeding it discards excess bytes and sets `output_limit_exceeded`. `stdout_paused` / `stderr_paused` report state.
- **Early Exit**: Supports a predicate that can stop consumption early (e.g., if a specific log line is detected). Checked on stdout only, per chunk; the matching chunk is retained.
- **PTY master**: When constructed with the `pty_master` flag, a **stdout** `EIO` is mapped to a clean EOF (the pty EOF signal); otherwise `EIO` surfaces as an error. A **stderr** `EIO` always surfaces as an error, even on pty spawns.
- **Stateless**: `DrainState` does not own the `Reactor`. It provides tokens and handlers that the caller must integrate into their own event loop.
## Process Spawning Lifecycle
`spawn::SpawnOptions::builder(argv, backend)` returns a `SpawnOptionsBuilder`; call `.build()` to get the `SpawnOptions`. Three entry points:
### Builder surface and defaults
`SpawnOptionsBuilder` (also constructible via `SpawnOptionsBuilder::new`):
`env`, `cwd`, `stdin`, `capture_stdout`, `capture_stderr`, `wait`,
`pgroup`, `session_containment`, `max_output`, `timeout_ms`, `kill_grace_ms`,
`cancel`, `fd_policy`, `early_exit`, `chunk_sink`, `pty`, `pty_with`.
Defaults: `wait=true`, `max_output=1 MiB`, `kill_grace_ms=2000`,
`cancel=CancelPolicy::Kill`, `fd_policy=CloexecOnly`.
- `CancelPolicy` — `None` (no signal on drop/cancel), `Graceful` (TERM),
`Kill` (default). `None` never signals; the wait gives up with partial output.
- `chunk_sink(F)` / `SinkResult` — attach a streaming sink
(`Fn(bool, &[u8]) -> SinkResult`, the bool is `is_stdout`). `SinkResult::Pause`
applies backpressure (lossless — the chunk is held and re-delivered on
resume); `Accept` consumes. **With a sink attached, `max_output` is ignored**
(streaming mode is unbounded) and `EOVERFLOW` is never returned.
- `SpawnOptions::run(self)` is equivalent to `spawn(self)`.
- `spawn_managed` requires `wait=true`; `spawn_start` rejects background I/O
capture without `wait` (detached children can't have captured output drained
by the caller).
### Blocking: `spawn::spawn(opts)`
Runs the child to completion on the caller's thread and returns `Output` (stdout/stderr parts, `ExitStatus`, timeout/cancel flags). `timeout_ms` bounds the wait; `kill_grace_ms` controls TERM→KILL escalation. `Output` carries `pid`, `status: Option<ExitStatus>` (`None` when `wait=false`), `stdout`, `stderr`, `timed_out`, `stdout_early_exited`, and `stdout_pending`/`stderr_pending` (unflushed chunk-sink tails). `EOVERFLOW` is returned when combined output exceeds `max_output` on the natural-completion path; a forced close (timeout/cancel with a wedged pipe) returns partial output with `timed_out` instead.
### Reactor-driven: `spawn::spawn_start(opts)`
Returns a `RunningProcess` handle the caller drives with their own `Reactor`:
```rust
use coreshift_core::spawn::{self, SpawnBackend, SpawnOptions};
use coreshift_core::reactor::Reactor;
fn example() -> Result<(), coreshift_core::CoreError> {
let mut reactor = Reactor::new()?;
let mut running = spawn::spawn_start(
SpawnOptions::builder(vec!["/bin/ls".to_string()], SpawnBackend::PosixSpawn)
.capture_stdout()
.build()?
)?;
running.register_with_reactor(&mut reactor)?;
let mut events = Vec::new();
while !running.io_done() {
reactor.wait(&mut events, 64, -1)?;
for ev in &events {
running.handle_reactor_event(&mut reactor, ev)?;
}
}
let (stdout, stderr) = running.into_output_parts();
Ok(())
}
```
`RunningProcess` exposes `resume_stdout`/`resume_stderr`, `resize_pty`, `write_input` (pty stdin), `stdout_paused`/`stderr_paused`/`io_done`, and `into_output_parts`. There is no `pid()` method — the child is `running.process` (a public `Process` field), so the pid is `running.process.pid()`.
### Managed: `spawn::spawn_managed(opts)`
Returns a `ManagedProcess` for the full non-blocking lifecycle: `register_with_reactor`, `poll_completion` (with `next_deadline()` for the next timeout), `request_cancel` (TERM→KILL escalation with `kill_grace_ms`), and `resize_pty`/`write_input`. Requires `wait=true`. Dropping an unfinished `ManagedProcess` signals per its `CancelPolicy` (SIGKILL under `CancelPolicy::Kill`, no signal under `CancelPolicy::None`), reaps it with a bounded wait (~100 ms), and hands an unreaped pid to the orphan reaper.
## PTY Spawns
`SpawnOptionsBuilder::pty()` spawns the child on a pseudo-terminal:
- `Pipes` opens `/dev/ptmx` (`grantpt`/`unlockpt`/`ptsname_r`); the child `setsid`s, dup2s the slave onto fd 0/1/2, and claims it with `TIOCSCTTY`.
- The master is drained as the child's single merged stdout+stderr stream.
- `resize_pty(rows, cols)` applies `TIOCSWINSZ`; `write_input` feeds stdin (bounded via `POLLOUT` at 2 s; `EINVAL` for non-pty spawns or a closed master).
- `make_pty()` / `pty_window()` expose the raw primitives for callers that build their own pty plumbing.
- PTY is rejected for `PosixSpawn`, without an isolated process group, or combined with a stdin buffer.
## Process Groups and Containment
`ProcessGroup::new(leader, isolated)` controls the child's session/pgroup. `isolated=true` runs the child in its own session (`setsid`). `SpawnOptionsBuilder::session_containment()` additionally installs a seccomp filter in the child (before `execve`) denying `setsid`, `setpgid`/`setpgrp`, `unshare`, and `setns` — locking the child and all descendants into the group the daemon placed them in, so `kill_group` is total even against a hostile root child. For **pty spawns** a separate filter is installed that still denies `setsid`/`unshare`/`setns` but **allows `setpgid`** (shell job control needs it). Requires an isolated process group; rejected on `PosixSpawn`.
`Process` (a running/reaped child handle) provides `wait_step`, `wait_blocking`, `kill`, `kill_pgroup`, `kill_group`, `pidfd` (when the backend delivered one), and guards against pid recycling.
## Spawn Backends
`SpawnBackend` is explicit and caller-chosen — Core never switches silently:
- `PosixSpawn` — `posix_spawn`, no child-setup step.
- `Fork` — `fork`/`exec`; the shared `child_entry` applies the selected `SpawnFdPolicy` in the child.
- `Vfork` — `vfork`/`exec`; shared address space until exec, async-signal-safe setup only.
- `Clone3` — `clone3`/`exec` with explicit clone flags (`ENOSYS` pre-5.3).
- `Clone3Pidfd` — as `Clone3`, plus a pidfd for the child (pid-reuse-immune signaling, pidfd-poll exit detection).
`SpawnFdPolicy` controls descriptor inheritance: `CloexecOnly` (default), `CloseFrom3`, `Allowlist`. `CloseFrom3`/`Allowlist` are enforced by **all four exec backends** (Fork/Vfork/Clone3/Clone3Pidfd) via the shared child entry (a `getdents64` scan of `/proc/self/fd` that closes every fd ≥ 3 not required or allowlisted). `PosixSpawn` only accepts `CloexecOnly`.
## Guarantees
- **No Hidden Threads**: All I/O occurs on the thread that calls `Reactor::wait` and the subsequent handler methods. One named exception exists (see ARCHITECTURE.md): the process-lifetime `spawn-orphan-reaper` thread, which reaps children the caller will never wait on (see above).
- **Memory Safety**: In accumulate mode, `DrainState` bounds captured output (combined stdout+stderr) at the configured limit; when the limit is exceeded it discards excess bytes and sets `output_limit_exceeded` rather than erroring. With a `ChunkSink` attached (streaming mode) the limit is **not** applied — the sink must enforce its own bounds.
- **Bounded Wait**: D-state (uninterruptible-sleep) children are bounded — SIGKILL is sent only while unreaped, and after a bound the spawn returns `timed_out` with partial output; the kernel reaps the child when it leaves D-state, otherwise the orphan reaper does.