task_runs/driver.rs
1//! @arch:layer(kg_store)
2//! @arch:role(substrate)
3//! @arch:see(.yah/docs/working/yah-task-runs.md)
4//!
5//! PTY subprocess driver — spawn commands, capture output as append-only
6//! chunks, handle SIGTERM/SIGKILL with a grace period, and mark stale
7//! `Running` runs as `Lost` when the daemon restarts.
8//!
9//! ## Tier 2 side-channel (yah-log shims)
10//!
11//! When `SpawnOpts::log_fd_enabled` is true (the default), the driver creates
12//! a named pipe (FIFO) and exports two env vars into the child:
13//!
14//! - `YAH_TASK_RUN` — the `TaskRunId` as a hyphenated UUID string.
15//! - `YAH_LOG_PIPE` — absolute path to the FIFO.
16//!
17//! The child opens `YAH_LOG_PIPE` for writing and emits JSON-lines. The
18//! driver reads those lines in a background thread and stores them as
19//! [`EventSource::Shim`] events.
20//!
21//! **Why FIFO instead of a raw fd?** `portable-pty` calls `close_random_fds()`
22//! in its `pre_exec` hook, closing every fd ≥ 3 before exec. A raw-pipe write
23//! fd is always ≥ 3 and would be closed before the child could use it. Opening
24//! a FIFO by path requires no fd inheritance.
25//!
26//! Wire format — one JSON object per line:
27//! ```json
28//! {"level":"info","target":"myapp::module","msg":"text","fields":{"key":"val"}}
29//! ```
30//! Optional shim-identity keys: `"_lib"` (string), `"_lib_ver"` (string).
31//! Unknown keys in `fields` pass through as freeform JSON.
32//!
33//! The driver holds the write end of the FIFO open until the run lifecycle
34//! task completes, which triggers EOF for the receiver thread. The FIFO file
35//! is deleted after the receiver thread drains the last line.
36//!
37//! On non-Unix platforms `YAH_TASK_RUN` and `YAH_LOG_PIPE` are not exported.
38//! Shim libraries must treat absent `YAH_TASK_RUN` as "not inside a TaskRun".
39//!
40//! @yah:ticket(R617-F6, "Reattach-by-run_id replaces Lost-on-disappear for origin=terminal shells")
41//! @yah:status(review)
42//! @yah:assignee(agent:bundle-anthropic-ashguard)
43//! @yah:at(2026-07-24T01:26:41Z)
44//! @yah:phase(P3)
45//! @yah:parent(R617)
46//! @arch:see(.yah/docs/working/W280-durable-terminal-sessions.md)
47//! @yah:depends_on(R617-F13)
48//! @yah:handoff("DELIVERED. Verified: `cd oss/qed && cargo test -p task-runs --lib` 243/243 (was 237 — 6 new); `cargo test -p kg-daemon --lib shell_vt` 9/9; `cargo test -p yah --lib r617` 9/9; `cargo test -p desktop --lib` 357 pass / 2 fail, both pre-existing and in files this ticket does not touch (agent.rs rules-view expects 12 rows and a peer's approval-rule change makes 19; agent_process reader-finished is a known timing flake).")
49//! @yah:handoff("THE TICKET'S OWN FRAMING WAS WRONG ABOUT THE MECHANISM, and the correction is the design. @yah:next said to 're-adopt' a live shell by 'control channel rebuilt, reader thread restarted against the surviving PTY'. That is not possible and never was: you cannot re-open another process's PTY master fd. The real defect is narrower and worse — a driver was tombstoning runs IT DID NOT OWN. `.yah/db/task-runs.turso` has several writers (desktop, the R617-F13 shell host, one CampService per MCP sidecar), and `TaskDriver::new` assumed any leftover `Running` row must be its own predecessor's corpse. So every attach marked some other LIVE process's shell `Lost`, and that shell kept producing output under a status saying it was dead. The fix is therefore 'do not tombstone what you do not own', not 'reattach'. Actual PTY reattach is unnecessary once F13 puts the PTY in a process that outlives the desktop.")
50//! @yah:handoff("HOW OWNERSHIP IS KNOWN: new `TaskRunMeta::host_pid` — the pid of the process whose driver spawned the run, NOT the child's. Stamped by `spawn_run` at INSERT, before the child exists, so a crash between insert and spawn still leaves the row attributable. Store column added by the same idempotent `ALTER TABLE ... ADD COLUMN` pattern `origin` used, and `row_to_meta` reads index 15 with `.ok().flatten()` so a DB with no such column reads `None` rather than erroring.")
51//! @yah:handoff("THE SEAM IS ORIGIN-AGNOSTIC, per this ticket's gotcha. New `task_runs::StaleRunPolicy` in oss/qed/crates/task-runs/src/driver.rs: `LostOnDisappear` (the default — `TaskDriver::new` and `with_channels` behave exactly as before, so no existing embedder changed) and `AdoptLiveHosts { origins: Vec<String> }`, which spares a leftover run only when its `host_pid` names a process that still exists. The crate decides on OWNERSHIP and takes the origin list as data — it never learns what 'terminal' means. New `TaskDriver::with_config` is the constructor that takes it.")
52//! @yah:handoff("yah side: `crates/yah/kg-daemon/src/service.rs::open_task_store` now passes `AdoptLiveHosts { origins: [ORIGIN_TERMINAL] }`. Also replaced the magic string — new `kg_daemon::shell_vt::ORIGIN_TERMINAL` now backs the two live `origin == \"terminal\"` gates in shell_vt.rs plus the policy, so the VT-parsing gate and the tombstone-exemption gate cannot drift apart by a typo. The constant lives on the yah side, NOT in task-runs, precisely to keep the crate generic.")
53//! @yah:handoff("Also stamped at app/yah/desktop/src/terminal.rs:519 — the desktop-local PTY path (terminal_open_local's scrollback mint) owns its own PTYs, so those rows carry the desktop's pid. Without it the shell host's driver would tombstone a live desktop-local session on attach, which is the same bug pointing the other way.")
54//! @yah:handoff("PID REUSE is the honest weakness and is why the policy is opt-in and origin-narrowed. `kill(pid, 0)` (EPERM counts as alive — the process exists, it is just not ours to signal) can read a recycled pid as the original owner. The failure mode of a false 'alive' is one run left `Running` until something closes it; the false 'dead' this replaces kills a live session's status. Strictly the better direction for an interactive shell, and the exposure is bounded to origins the embedder opted in. Non-unix has no kill(2), so `host_process_alive` reports false there and the platform keeps the old behaviour rather than stranding runs forever.")
55//! @yah:handoff("SIX NEW TESTS, each pinned to a failure rather than a code path: a live-owner terminal run survives a new driver (the ticket's whole point); a run whose owner pid was spawned and reaped in-test IS tombstoned (a crashed host must not leave zombie tiles); origin-less and non-matching origins are tombstoned even with a live owner (an in-flight `cargo build` whose driver is gone has nobody left to record its exit); an unattributed row (pre-migration) is tombstoned; `TaskDriver::new` still tombstones unconditionally (no silent behaviour change for existing embedders); and `spawn_run` stamps this process — the policy is worthless if rows arrive unattributed.")
56//! @yah:verify("cd oss/qed && cargo test -p task-runs --lib # 243/243, 6 new under driver::tests")
57//! @yah:verify("cargo test -p kg-daemon --lib shell_vt # 9/9")
58//! @yah:verify("cargo test -p yah --lib r617 # 9/9")
59//! @yah:verify("Manual (needs a desktop rebuild): open a shell, run `sleep 300`, quit and relaunch the desktop — the run is still Running, not Lost")
60//! @yah:verify("sqlite3 .yah/db/task-runs.turso \"select id, origin, host_pid, status from runs where status='running';\" # every live row names a pid that ps shows")
61//! @yah:gotcha("This is an oss/qed crate — changes land in-tree under oss/qed/crates/task-runs and flow outward via scripts/export-oss.sh. The seam was kept origin-agnostic (StaleRunPolicy decides on host_pid, takes origins as data); the one yah-ism, ORIGIN_TERMINAL, lives in crates/yah/kg-daemon/src/shell_vt.rs instead.")
62//! @yah:gotcha("`host_pid` is NOT on the wire. rpc::WireRunMeta does not carry it, so a client cannot ask 'is this run's owner alive'. Nothing needs it today — the policy runs entirely daemon-side — but R617-F7 should check whether reattaching tiles want it before adding a second liveness notion of their own.")
63//! @yah:gotcha("pid reuse can make a dead owner read alive, leaving a run `Running` with nobody driving it. Bounded on purpose (opt-in + origin-narrowed) and strictly safer than the false-dead it replaces, but it is a real edge: if zombie terminal rows ever accumulate, this is why.")
64//! @yah:gotcha("TaskRunMeta gained a required field, so every struct-literal construction site had to be updated (velveteen-exec x4, scryer, task-runs fixtures, kg-daemon fixtures, desktop/terminal.rs x2). A new construction site added by anyone else will fail to compile until they pick a value — which is the intended forcing function: a run with no recorded owner is a run the policy has to tombstone.")
65//!
66//! @yah:ticket(R617-B9, "Pre-existing: task-runs log_pipe_events_land_in_store never completes (233 pass / 1 fail)")
67//! @yah:status(review)
68//! @yah:assignee(agent:bundle-anthropic-ashguard)
69//! @yah:at(2026-07-22T19:50:25Z)
70//! @yah:phase(P1)
71//! @yah:parent(R617)
72//! @yah:handoff("Root cause: not the FIFO, not the PTY. The whole pipeline completed correctly every time (child wrote the JSON line, receiver drained it, reader hit EOF, child.wait returned 0) — but the lifecycle's terminal `store.update_status` returned `Sql(Busy(\"database is locked\"))` and run_lifecycle swallowed it with `let _ =`, so the run stayed Running forever and the 20s poll deadline blew. A live run has three concurrent turso writers (PTY chunk appends, shim-FIFO event appends, lifecycle status) on independent connections with no busy handling at all.")
73//! @yah:handoff("Fix in oss/qed/crates/task-runs/src/store.rs: (1) `conn()` now sets `busy_timeout(5s)` on every connection; (2) new `exec_retry()` wraps writes in an outer exponential-backoff retry on the `Busy`/`BusySnapshot` class, because turso caps its internal backoff and then hands `Busy` back; (3) insert_run / update_status / update_beholder_status / append_chunk / append_event all routed through it.")
74//! @yah:handoff("driver.rs run_lifecycle no longer swallows the terminal status write — a genuine failure after retries now prints `[yah task-runs] failed to record terminal status for run <id>`, matching the crate's existing eprintln convention.")
75//! @yah:handoff("New regression test store.rs::concurrent_writers_do_not_lose_the_terminal_status — two background tasks hammer append_chunk/append_event while update_status lands. Verified it has teeth: with busy_timeout and the retry disabled it fails 3/3 with the exact `Busy(\"database is locked\")`; with them it passes 5/5.")
76//! @yah:verify("cd oss/qed && cargo test -p task-runs --lib — 237 passed / 0 failed (was 235 pass / 1 fail)")
77//! @yah:verify("log_pipe_events_land_in_store run 8x sequentially: 8/8 green in ~0.58s each. Before the fix the same loop was 11/12 red at the 20s timeout.")
78//!
79//! @yah:ticket(R652-T6, "Login shell: when cmd is the resolved shell, exec it directly (not sh -c) with -l")
80//! @yah:at(2026-08-02T00:03:08Z)
81//! @yah:status(review)
82//! @yah:assignee(agent:bundle-ollama-cloud-boulder)
83//! @yah:phase(P1)
84//! @yah:parent(R652)
85//! @yah:handoff("Login shells now exec directly with -l instead of going through sh -c. SpawnOpts (oss/qed/crates/task-runs/src/driver.rs) gained `argv: Option<Vec<String>>`: when set, spawn_run builds the CommandBuilder from that argv verbatim instead of wrapping `cmd` in `sh -c`. camp-service task_run sets it to [resolved_shell, \"-l\"] whenever the request is a shell request.")
86//! @yah:handoff("Why an argv escape hatch rather than a `login_shell: bool` flag in the driver: task-runs is an oss/qed crate and has no business knowing what a login shell is. The caller names the exact process; the driver just execs it. This also made R652-T4 a two-line addition rather than a second flag.")
87//! @yah:handoff("Three things this fixes beyond .zprofile finally running. (1) `sh -c \"zsh -l\"` left an inert `sh` as the PTY's foreground process group leader, so job control misbehaved and signals went to the wrong process. (2) That same inert sh is what the foreground-pid cwd probe (R652-T2) would have reported for, so T2 could not have worked without this. (3) -l is now a real argv element instead of text inside a shell string, so no quoting layer can eat it.")
88//! @yah:handoff("`cmd` is still what lands on TaskRunMeta.command, so a shell run reads back as \"$SHELL\" -- the rail label and the history re-run path both keep working. Beholder argv rewriting is bypassed when argv is set (the attach runs with BeholderSelect::None): the rewritten argv would be discarded on that path, so recording a `rewrite=...` that never happened would be a lie in the run metadata.")
89//! @yah:handoff("An empty argv falls back to the sh -c path rather than spawning nothing -- a caller bug should not become an exec of the empty string.")
90//! @yah:verify("cd oss/qed && cargo test -p task-runs --lib # 246/246 green (3 new: explicit_argv_execs_the_program_directly, explicit_argv_still_records_the_requested_command, empty_argv_falls_back_to_the_shell_path)")
91//! @yah:verify("Manual (needs desktop rebuild): add `echo W289-login-test >> /tmp/w289.log` to ~/.zprofile, open a shell tile, confirm the file gets a line")
92//! @yah:gotcha("driver.rs is an oss/qed crate -- this lands in-tree under oss/qed/crates/task-runs and flows outward via scripts/export-oss.sh on the next release. SpawnOpts gained a field, but every in-tree construction site uses ..Default::default(), so nothing else needed touching.")
93
94use std::collections::HashMap;
95use std::io::Read;
96use std::path::PathBuf;
97use std::sync::{Arc, Mutex};
98use std::time::{Duration, SystemTime, UNIX_EPOCH};
99
100use portable_pty::{native_pty_system, CommandBuilder, PtySize};
101use thiserror::Error;
102use tokio::sync::{mpsc, oneshot};
103use tokio::task;
104
105use crate::beholders::{registry_with_user_beholders, BeholderSelect};
106use crate::store::{RunFilter, StoreError, TaskStore};
107use crate::types::{BeholderStatus, Initiator, OutputChunk, RunStatus, Stream, TaskRunId, TaskRunMeta};
108
109const DEFAULT_GRACE: Duration = Duration::from_secs(5);
110const READ_BUF_SIZE: usize = 4096;
111const SIGTERM: i32 = 15;
112const SIGKILL: i32 = 9;
113
114// ─── Error ────────────────────────────────────────────────────────────────────
115
116#[derive(Debug, Error)]
117pub enum DriverError {
118 #[error("store: {0}")]
119 Store(#[from] StoreError),
120 #[error("pty: {0}")]
121 Pty(String),
122 #[error("run not found: {0}")]
123 NotFound(String),
124 #[error("io: {0}")]
125 Io(#[from] std::io::Error),
126}
127
128// ─── SpawnOpts ────────────────────────────────────────────────────────────────
129
130/// Options for [`TaskDriver::spawn_run`].
131#[derive(Debug, Clone)]
132pub struct SpawnOpts {
133 pub cwd: PathBuf,
134 /// Env vars set on the child process (merged on top of the current env).
135 pub env: Vec<(String, String)>,
136 pub label: Option<String>,
137 pub initiator: Initiator,
138 /// PTY column count. Defaults to 80.
139 pub pty_cols: u16,
140 /// PTY row count. Defaults to 24.
141 pub pty_rows: u16,
142 /// Enable stdin relay via [`TaskDriver::send_stdin`].
143 pub stdin_enabled: bool,
144 /// Pin the run so the GC sweep does not drop its output during warm rolloff.
145 pub pin: bool,
146 /// Beholder attachment policy. Defaults to [`BeholderSelect::Auto`].
147 pub beholder_select: BeholderSelect,
148 /// `true` when a human-facing terminal tile is attached. Causes `Rewriter`
149 /// beholders to decline in `Auto` mode so the human sees unmodified output.
150 pub tty_attached: bool,
151 /// Create a side-channel FIFO and export `YAH_TASK_RUN` / `YAH_LOG_PIPE`
152 /// so Tier-2 shim libraries (yah-log-rust, @yah/log) can emit structured
153 /// events. Has no effect on non-Unix platforms. Defaults to `true`.
154 pub log_fd_enabled: bool,
155 /// Provenance tag stored on the run's `TaskRunMeta.origin` (e.g.
156 /// `Some("terminal")` for an interactive shell). `None` is an ordinary job.
157 pub origin: Option<String>,
158 /// Exec this argv directly instead of wrapping `cmd` in `sh -c`.
159 ///
160 /// The default `sh -c <cmd>` is right for a job — the caller wrote a
161 /// command line and expects a shell to parse it. It is wrong for an
162 /// *interactive shell*: `sh -c "zsh -l"` leaves an inert `sh` as the PTY's
163 /// foreground process group leader, so job control misbehaves, signals go
164 /// to the wrong process, and anything that reads the foreground pid (a
165 /// live-cwd probe, say) sees `sh` instead of the shell the operator is
166 /// typing into. Handing the exact argv here makes the shell itself the
167 /// child, which is also the only way to pass `-l` as a real argv element
168 /// so `.zprofile` / `.profile` actually run.
169 ///
170 /// `cmd` is still what gets recorded on `TaskRunMeta.command`, so the run
171 /// reads the way the caller asked for it. Beholder argv rewriting is
172 /// bypassed when this is set: the caller has already decided the exact
173 /// process to exec, and a recorded `rewrite=…` that didn't happen would be
174 /// a lie in the run metadata.
175 pub argv: Option<Vec<String>>,
176}
177
178impl Default for SpawnOpts {
179 fn default() -> Self {
180 Self {
181 cwd: std::env::current_dir().unwrap_or_else(|_| PathBuf::from("/")),
182 env: vec![],
183 label: None,
184 initiator: Initiator::Human { camp: "local".to_string() },
185 pty_cols: 80,
186 pty_rows: 24,
187 stdin_enabled: false,
188 pin: false,
189 beholder_select: BeholderSelect::Auto,
190 tty_attached: false,
191 log_fd_enabled: true,
192 origin: None,
193 argv: None,
194 }
195 }
196}
197
198// ─── Driver channels ─────────────────────────────────────────────────────────
199
200/// Optional side-channels a driver can publish to. Both are fire-and-forget:
201/// a closed receiver never stalls or fails a run.
202#[derive(Default)]
203pub struct DriverChannels {
204 /// Fires `(run_id, status)` after each run's lifecycle task writes the
205 /// terminal status. Drives completion listeners (e.g. a triage worker).
206 pub completion: Option<mpsc::UnboundedSender<(TaskRunId, RunStatus)>>,
207 /// Mirrors every PTY output chunk as it is captured, *before* any consumer
208 /// polls the store. Lets a host attach a live view (VT parser, log
209 /// forwarder) to a run without a read-back loop over the store.
210 ///
211 /// The driver deliberately stays ignorant of what the tap is for — the
212 /// chunk carries `run_id`, so the host decides which runs it cares about.
213 pub output: Option<mpsc::UnboundedSender<OutputChunk>>,
214}
215
216// ─── Stale-run policy ────────────────────────────────────────────────────────
217
218/// What a freshly-constructed [`TaskDriver`] does with `Running` rows it finds
219/// already in the store.
220///
221/// The historical rule — tombstone every one of them — bakes in an assumption
222/// that stops being true the moment a second process attaches to the same
223/// store: that any `Running` row must be a corpse from *this* process's
224/// predecessor. When two processes share a store, a driver starting up in one
225/// will happily mark the other's live runs `Lost`, and the run keeps producing
226/// output under a status that says it is dead.
227///
228/// The policy is deliberately origin-agnostic in its mechanism — it decides on
229/// **who owns the run** ([`TaskRunMeta::host_pid`]) — and takes the origin list
230/// as data, so an embedder names the runs it wants exempted without this crate
231/// knowing what any of them mean.
232#[derive(Debug, Clone, Default, PartialEq, Eq)]
233pub enum StaleRunPolicy {
234 /// Tombstone every leftover `Running` run as `Lost`.
235 ///
236 /// Correct, and the default, whenever this process is the only writer:
237 /// a run whose driver is gone has no one left to notice it exit.
238 #[default]
239 LostOnDisappear,
240 /// Spare runs whose recorded owner process is still alive.
241 ///
242 /// A leftover run is tombstoned only when its `host_pid` is absent (owner
243 /// unknown — a row from before the column existed) or names a process that
244 /// no longer exists. Anything else belongs to a live peer and is left
245 /// `Running` for that peer to finish.
246 ///
247 /// `origins` narrows the exemption to runs whose
248 /// [`TaskRunMeta::origin`] is in the list; empty means every origin
249 /// qualifies. A run with no origin never matches a non-empty list.
250 AdoptLiveHosts { origins: Vec<String> },
251}
252
253impl StaleRunPolicy {
254 /// Whether `meta` should be tombstoned `Lost` at driver construction.
255 fn tombstones(&self, meta: &TaskRunMeta) -> bool {
256 match self {
257 StaleRunPolicy::LostOnDisappear => true,
258 StaleRunPolicy::AdoptLiveHosts { origins } => {
259 let exempt_origin = origins.is_empty()
260 || meta
261 .origin
262 .as_deref()
263 .is_some_and(|o| origins.iter().any(|want| want == o));
264 if !exempt_origin {
265 return true;
266 }
267 match meta.host_pid {
268 Some(pid) => !host_process_alive(pid),
269 None => true,
270 }
271 }
272 }
273 }
274}
275
276/// Is a process with this pid still around?
277///
278/// `kill(pid, 0)` is the portable liveness probe: it performs the permission
279/// check and existence lookup without delivering anything. `EPERM` counts as
280/// alive — the process exists, it just is not ours to signal.
281///
282/// Pid reuse can make a dead owner read as alive. That is why
283/// [`StaleRunPolicy::AdoptLiveHosts`] is opt-in and origin-narrowed: the cost
284/// of a false "alive" is one run left `Running` until something closes it,
285/// which is strictly better for an interactive session than the false "dead"
286/// this replaces — which kills a *live* session's status.
287#[cfg(unix)]
288fn host_process_alive(pid: u32) -> bool {
289 if pid == 0 {
290 return false;
291 }
292 if pid == std::process::id() {
293 return true;
294 }
295 // SAFETY: `kill` with signal 0 delivers nothing; it only reports whether
296 // the pid exists and is signallable.
297 let rc = unsafe { libc::kill(pid as libc::pid_t, 0) };
298 rc == 0 || std::io::Error::last_os_error().raw_os_error() == Some(libc::EPERM)
299}
300
301/// No `kill(2)` off Unix. Reporting every owner dead keeps the historical
302/// Lost-on-disappear behaviour rather than stranding runs `Running` forever.
303#[cfg(not(unix))]
304fn host_process_alive(_pid: u32) -> bool {
305 false
306}
307
308// ─── Internal run-control handle ─────────────────────────────────────────────
309
310struct RunControl {
311 kill_tx: mpsc::Sender<KillRequest>,
312 stdin_tx: Option<mpsc::Sender<Vec<u8>>>,
313 /// Shared with the lifecycle task, which holds the same `Arc` so the PTY fd
314 /// outlives `child.wait()`. `MasterPty::resize` takes `&self`, so a mutex is
315 /// enough to make the `Box<dyn MasterPty + Send>` `Sync` across the two.
316 master: Arc<Mutex<Box<dyn portable_pty::MasterPty + Send>>>,
317}
318
319#[derive(Debug)]
320struct KillRequest {
321 signal: i32,
322}
323
324// ─── ShimRecord ───────────────────────────────────────────────────────────────
325
326/// One JSON-line record emitted by a Tier-2 shim to the side-channel FIFO.
327///
328/// The shim (Rust `yah-log` layer or TS `@yah/log` pino transport) writes one
329/// of these per log call. Unknown keys inside `fields` pass through unchanged.
330#[cfg(unix)]
331#[derive(serde::Deserialize)]
332struct ShimRecord {
333 level: String,
334 target: String,
335 msg: String,
336 #[serde(default)]
337 fields: serde_json::Value,
338 /// Shim library name, e.g. `"yah-log-rust"`. Populates
339 /// [`EventSource::Shim::lib`].
340 #[serde(rename = "_lib", default)]
341 lib: Option<String>,
342 /// Shim library version string.
343 #[serde(rename = "_lib_ver", default)]
344 lib_version: Option<String>,
345}
346
347// ─── FdCloser ─────────────────────────────────────────────────────────────────
348
349/// RAII wrapper that closes a raw fd on drop.
350///
351/// Used to hold the write end of the log FIFO open until the lifecycle task
352/// completes. Dropping it signals EOF to the receiver thread.
353#[cfg(unix)]
354struct FdCloser(libc::c_int);
355
356#[cfg(unix)]
357impl Drop for FdCloser {
358 fn drop(&mut self) {
359 unsafe { libc::close(self.0) };
360 }
361}
362
363// SAFETY: a raw fd number is an integer; closing it from any thread is safe
364// provided we never duplicate ownership (enforced by move semantics here).
365#[cfg(unix)]
366unsafe impl Send for FdCloser {}
367
368// ─── TaskDriver ───────────────────────────────────────────────────────────────
369
370/// Manages in-flight task runs for a single camp.
371///
372/// Wrap in `Arc` to share across tasks; internal state is mutex-protected.
373pub struct TaskDriver {
374 store: Arc<TaskStore>,
375 active: Arc<Mutex<HashMap<String, RunControl>>>,
376 /// Side-channels published to by every run this driver owns.
377 channels: DriverChannels,
378}
379
380impl TaskDriver {
381 /// Create a driver backed by `store`, with no side-channels.
382 ///
383 /// Immediately scans the store for `Running` runs left over from a prior
384 /// daemon process and marks them `Lost` ("Lost-on-disappear").
385 pub async fn new(store: Arc<TaskStore>) -> Result<Self, DriverError> {
386 Self::with_channels(store, DriverChannels::default()).await
387 }
388
389 /// Like `new` but wires the optional [`DriverChannels`] side-channels
390 /// (completion notifications, live output tap).
391 pub async fn with_channels(
392 store: Arc<TaskStore>,
393 channels: DriverChannels,
394 ) -> Result<Self, DriverError> {
395 Self::with_config(store, channels, StaleRunPolicy::default()).await
396 }
397
398 /// Full constructor: side-channels plus the [`StaleRunPolicy`] applied to
399 /// `Running` rows already in the store.
400 ///
401 /// R617-F6 — annotation in this file's header. Splitting the sweep out of
402 /// the constructor's fixed behaviour is what lets a store be shared: a
403 /// process that is not the run's owner can now attach without declaring
404 /// the owner's live work dead.
405 pub async fn with_config(
406 store: Arc<TaskStore>,
407 channels: DriverChannels,
408 stale_policy: StaleRunPolicy,
409 ) -> Result<Self, DriverError> {
410 let stale = store
411 .list_runs(&RunFilter {
412 status: Some("running".to_string()),
413 ..Default::default()
414 })
415 .await?;
416 for meta in stale {
417 if !stale_policy.tombstones(&meta) {
418 continue;
419 }
420 store
421 .update_status(
422 &meta.id,
423 &RunStatus::Lost {
424 reason: "daemon restarted while run was in-flight".to_string(),
425 },
426 )
427 .await?;
428 }
429 Ok(Self {
430 store,
431 active: Arc::new(Mutex::new(HashMap::new())),
432 channels,
433 })
434 }
435
436 /// Spawn `cmd` in a PTY and start capturing its output. Returns immediately
437 /// with the new [`TaskRunId`].
438 ///
439 /// A beholder is selected via `opts.beholder_select` (default `Auto`). When
440 /// a `Rewriter` beholder matches, its `adjust_argv` is applied to the
441 /// command before spawning and the diff is recorded on `beholder_status`.
442 /// When `opts.tty_attached` is `true`, `Rewriter` beholders decline in
443 /// `Auto` mode to preserve human-readable output.
444 ///
445 /// Output is written to the store as `Stream::Stdout` chunks (the PTY
446 /// kernel merges stdout and stderr). Signal handling and status updates
447 /// run in background tasks.
448 pub async fn spawn_run(&self, cmd: &str, opts: SpawnOpts) -> Result<TaskRunId, DriverError> {
449 let id = TaskRunId::new();
450 let started_at = unix_now_secs();
451 let started_at_ms: u64 = started_at.saturating_mul(1000);
452
453 // Attach a beholder (may rewrite argv and produce structured events).
454 // Resolve user drop-in directory: $YAH_BEHOLDERS_DIR or $HOME/.yah/beholders.
455 let user_dir = std::env::var_os("YAH_BEHOLDERS_DIR")
456 .map(std::path::PathBuf::from)
457 .or_else(|| {
458 std::env::var_os("HOME")
459 .map(|h| std::path::PathBuf::from(h).join(".yah/beholders"))
460 });
461 let registry = registry_with_user_beholders(user_dir.as_deref());
462 /* An explicit argv means the caller already chose the exact process
463 (an interactive login shell, say). Selecting a beholder there would
464 either do nothing — the rewritten argv is discarded on that path —
465 or record a rewrite that never happened, so we opt out honestly
466 instead. */
467 let select = if opts.argv.is_some() {
468 &BeholderSelect::None
469 } else {
470 &opts.beholder_select
471 };
472 let attach = registry.attach(cmd, select, opts.tty_attached);
473 // Use the (possibly rewritten) argv to reconstruct the effective command.
474 let effective_cmd = if attach.argv.is_empty() {
475 cmd.to_string()
476 } else {
477 attach.argv.join(" ")
478 };
479
480 self.store.insert_run(&TaskRunMeta {
481 id: id.clone(),
482 command: cmd.to_string(),
483 cwd: opts.cwd.clone(),
484 env: opts.env.clone(),
485 started_at,
486 status: RunStatus::Running,
487 label: opts.label.clone(),
488 initiator: opts.initiator.clone(),
489 beholder_status: Some(attach.status),
490 pinned: opts.pin,
491 origin: opts.origin.clone(),
492 /* R617-F6: stamp the OWNER, before the child exists. Written at
493 insert rather than after spawn so a crash between the two still
494 leaves the row attributable — an unattributed `Running` row is
495 exactly what the conservative arm of `StaleRunPolicy` has to
496 tombstone. */
497 host_pid: Some(std::process::id()),
498 }).await?;
499
500 // Open PTY pair.
501 let pty_sys = native_pty_system();
502 let pair = pty_sys
503 .openpty(PtySize {
504 rows: opts.pty_rows,
505 cols: opts.pty_cols,
506 pixel_width: 0,
507 pixel_height: 0,
508 })
509 .map_err(|e| DriverError::Pty(e.to_string()))?;
510
511 // Clone reader before spawning so the fd is ready immediately.
512 let pty_reader = pair
513 .master
514 .try_clone_reader()
515 .map_err(|e| DriverError::Pty(e.to_string()))?;
516
517 // Optional stdin relay: take the writer before spawning the child.
518 let stdin_tx: Option<mpsc::Sender<Vec<u8>>> = if opts.stdin_enabled {
519 let mut writer = pair
520 .master
521 .take_writer()
522 .map_err(|e| DriverError::Pty(e.to_string()))?;
523 let (tx, mut rx) = mpsc::channel::<Vec<u8>>(64);
524 task::spawn(async move {
525 use std::io::Write;
526 while let Some(bytes) = rx.recv().await {
527 let _ = writer.write_all(&bytes);
528 let _ = writer.flush();
529 }
530 });
531 Some(tx)
532 } else {
533 None
534 };
535
536 // ── Side-channel log FIFO (Tier 2 / yah-log shims) ──────────────────
537 //
538 // Create a named pipe (FIFO) so child processes can write structured
539 // events without touching stdout/stderr. We export its path via
540 // YAH_LOG_PIPE; no fd inheritance is involved, so portable-pty's
541 // close_random_fds() pre_exec hook doesn't interfere.
542 //
543 // The parent opens the FIFO twice:
544 // rfd — O_RDONLY|O_NONBLOCK, then cleared to blocking → read events
545 // wfd — O_WRONLY (wrapped in FdCloser) → keeps the FIFO alive until
546 // the lifecycle task drops it (after run completion), producing
547 // EOF for the receiver thread.
548 #[cfg(unix)]
549 let log_fifo: Option<(libc::c_int, FdCloser, std::path::PathBuf)> = if opts.log_fd_enabled {
550 let fifo_path = std::env::temp_dir().join(format!("yah-log-{}.fifo", id));
551 let path_cstr = match std::ffi::CString::new(fifo_path.to_string_lossy().as_bytes()) {
552 Ok(s) => s,
553 Err(_) => {
554 // Path contained a nul byte — extremely unlikely; skip FIFO.
555 return Err(DriverError::Io(std::io::Error::new(
556 std::io::ErrorKind::InvalidInput,
557 "log FIFO path contained nul byte",
558 )));
559 }
560 };
561 let mkfifo_ret = unsafe { libc::mkfifo(path_cstr.as_ptr(), 0o600) };
562 if mkfifo_ret != 0 {
563 None // FIFO creation failed; continue without side-channel
564 } else {
565 // Open read end without blocking (no writer yet).
566 let rfd = unsafe {
567 libc::open(path_cstr.as_ptr(), libc::O_RDONLY | libc::O_NONBLOCK)
568 };
569 if rfd < 0 {
570 let _ = unsafe { libc::unlink(path_cstr.as_ptr()) };
571 None
572 } else {
573 // Switch read end to blocking so reads yield proper data.
574 unsafe { libc::fcntl(rfd, libc::F_SETFL, 0) };
575 // Open write end — this succeeds immediately because rfd is open.
576 let wfd = unsafe {
577 libc::open(path_cstr.as_ptr(), libc::O_WRONLY)
578 };
579 if wfd < 0 {
580 unsafe { libc::close(rfd) };
581 let _ = unsafe { libc::unlink(path_cstr.as_ptr()) };
582 None
583 } else {
584 Some((rfd, FdCloser(wfd), fifo_path))
585 }
586 }
587 }
588 } else {
589 None
590 };
591
592 // Build and spawn the child inside the slave. An explicit argv execs
593 // that program directly; otherwise the command line goes through `sh`
594 // so the caller's quoting, pipes and redirections mean what they say.
595 let mut cb = match opts.argv.as_deref() {
596 Some([program, args @ ..]) => {
597 let mut cb = CommandBuilder::new(program);
598 cb.args(args);
599 cb
600 }
601 // An empty argv is a caller bug, not a request for an empty exec —
602 // fall back to the shell path rather than spawning nothing.
603 _ => {
604 let mut cb = CommandBuilder::new("sh");
605 cb.args(["-c", &effective_cmd]);
606 cb
607 }
608 };
609 cb.cwd(&opts.cwd);
610 for (k, v) in &opts.env {
611 cb.env(k, v);
612 }
613 cb.env("TERM", "xterm-256color");
614
615 // Export YAH_TASK_RUN and YAH_LOG_PIPE if the FIFO was created.
616 #[cfg(unix)]
617 if let Some((_, _, ref fifo_path)) = log_fifo {
618 cb.env("YAH_TASK_RUN", id.to_string());
619 cb.env("YAH_LOG_PIPE", fifo_path.to_string_lossy().as_ref());
620 }
621
622 let child = pair
623 .slave
624 .spawn_command(cb)
625 .map_err(|e| DriverError::Pty(e.to_string()))?;
626 // Drop the parent's slave handle so EOF propagates once the child exits.
627 drop(pair.slave);
628
629 // Share the master between the lifecycle task (which must outlive
630 // `child.wait()` so the fd stays open) and `resize_run`.
631 let master: Arc<Mutex<Box<dyn portable_pty::MasterPty + Send>>> =
632 Arc::new(Mutex::new(pair.master));
633
634 let pid = child.process_id().unwrap_or(0);
635
636 // ── FIFO: launch receiver thread; pass write-end holder to lifecycle ──
637 //
638 // The receiver thread reads until EOF. EOF arrives when ALL write-end
639 // holders close: the child's own writers (when it exits) plus the
640 // FdCloser we hand to the lifecycle task (which drops it after writing
641 // the terminal RunStatus). Events written before the last close are
642 // still drained by the receiver thread before it exits.
643 #[cfg(unix)]
644 let log_wfd_holder: Option<FdCloser> = if let Some((rfd, wfd, fifo_path)) = log_fifo {
645 let store_log = Arc::clone(&self.store);
646 let id_log = id.clone();
647 let rt = tokio::runtime::Handle::current();
648 // spawn_blocking: lets the runtime track this thread so the
649 // Handle::block_on calls inside have a worker to drive futures.
650 tokio::task::spawn_blocking(move || {
651 run_log_receiver(rt, store_log, id_log, rfd, fifo_path, started_at_ms);
652 });
653 Some(wfd)
654 } else {
655 None
656 };
657
658 // Channels.
659 let (kill_tx, kill_rx) = mpsc::channel::<KillRequest>(4);
660 let (reader_done_tx, reader_done_rx) = oneshot::channel::<()>();
661
662 // Reader thread: PTY output → store chunks → beholder events.
663 // Runs on a dedicated OS thread because PTY reads are blocking.
664 {
665 let store_r = Arc::clone(&self.store);
666 let id_r = id.clone();
667 let mut beholder = attach.beholder;
668 let output_tx = self.channels.output.clone();
669 let rt = tokio::runtime::Handle::current();
670 tokio::task::spawn_blocking(move || {
671 let mut buf = [0u8; READ_BUF_SIZE];
672 let mut reader = pty_reader;
673 loop {
674 match reader.read(&mut buf) {
675 Ok(0) | Err(_) => break,
676 Ok(n) => {
677 let offset = elapsed_ms(started_at_ms);
678 let append_res = rt.block_on(store_r.append_chunk(
679 &id_r,
680 offset,
681 Stream::Stdout,
682 &buf[..n],
683 ));
684 if let Ok(seq) = append_res {
685 /* Both the tap and the beholder want the same
686 owned chunk; build it once, and only when
687 someone is listening. */
688 let chunk = (output_tx.is_some() || beholder.is_some()).then(|| {
689 OutputChunk {
690 run_id: id_r.clone(),
691 seq,
692 offset_ms: offset,
693 stream: Stream::Stdout,
694 bytes: buf[..n].to_vec(),
695 }
696 });
697 /* Tap first: it feeds live views, where latency
698 is visible to a human. Send failure means the
699 host dropped its receiver — never fatal. */
700 if let (Some(tx), Some(c)) = (&output_tx, &chunk) {
701 let _ = tx.send(c.clone());
702 }
703 let mut detach_beholder = false;
704 if let (Some(b), Some(chunk)) = (beholder.as_mut(), &chunk) {
705 for ev in b.parse_chunk(chunk) {
706 let _ = rt.block_on(store_r.append_event(
707 &ev.run_id,
708 ev.offset_ms,
709 ev.level,
710 &ev.target,
711 &ev.msg,
712 &ev.fields,
713 ev.anchor.as_ref().map(|a| a.seq),
714 &ev.source,
715 ));
716 }
717 if let Some(reason) = b.unknown_format_reason() {
718 let new_status = BeholderStatus::unknown_format_with_reason(
719 b.name(),
720 reason,
721 );
722 let _ = rt.block_on(
723 store_r.update_beholder_status(&id_r, &new_status),
724 );
725 detach_beholder = true;
726 }
727 }
728 if detach_beholder {
729 beholder = None;
730 }
731 }
732 }
733 }
734 }
735 if let Some(ref mut b) = beholder {
736 let final_offset = elapsed_ms(started_at_ms);
737 for ev in b.on_done(&id_r, final_offset) {
738 let _ = rt.block_on(store_r.append_event(
739 &ev.run_id,
740 ev.offset_ms,
741 ev.level,
742 &ev.target,
743 &ev.msg,
744 &ev.fields,
745 ev.anchor.as_ref().map(|a| a.seq),
746 &ev.source,
747 ));
748 }
749 if let Some(reason) = b.unknown_format_reason() {
750 let new_status = BeholderStatus::unknown_format_with_reason(b.name(), reason);
751 let _ = rt.block_on(store_r.update_beholder_status(&id_r, &new_status));
752 }
753 }
754 let _ = reader_done_tx.send(());
755 });
756 }
757
758 // Lifecycle task: monitor kill requests, wait for exit, update status.
759 // The task also holds the log FIFO write-end closer (if any) so that
760 // EOF propagates to the receiver thread after RunStatus is written.
761 {
762 let store_l = Arc::clone(&self.store);
763 let active_l = Arc::clone(&self.active);
764 let id_l = id.clone();
765 let master_l = Arc::clone(&master);
766 let completion_tx_l = self.channels.completion.clone();
767 #[cfg(unix)]
768 let wfd_l = log_wfd_holder;
769 task::spawn(async move {
770 run_lifecycle(
771 store_l,
772 active_l,
773 id_l,
774 pid,
775 child,
776 master_l,
777 kill_rx,
778 reader_done_rx,
779 completion_tx_l,
780 #[cfg(unix)]
781 wfd_l,
782 )
783 .await;
784 });
785 }
786
787 self.active
788 .lock()
789 .unwrap()
790 .insert(id.to_string(), RunControl { kill_tx, stdin_tx, master });
791
792 Ok(id)
793 }
794
795 /// Resize a running task's PTY and deliver `SIGWINCH` to the foreground
796 /// process group (portable-pty's `resize` does the ioctl, which is what
797 /// signals the child).
798 ///
799 /// Returns `DriverError::NotFound` when the run is not active on this
800 /// driver instance — the same contract as [`TaskDriver::send_stdin`].
801 pub async fn resize_run(
802 &self,
803 id: &TaskRunId,
804 cols: u16,
805 rows: u16,
806 ) -> Result<(), DriverError> {
807 let master = self
808 .active
809 .lock()
810 .unwrap()
811 .get(&id.to_string())
812 .map(|c| Arc::clone(&c.master));
813
814 match master {
815 Some(m) => {
816 let size = PtySize { rows, cols, pixel_width: 0, pixel_height: 0 };
817 m.lock()
818 .unwrap()
819 .resize(size)
820 .map_err(|e| DriverError::Pty(e.to_string()))
821 }
822 None => Err(DriverError::NotFound(id.to_string())),
823 }
824 }
825
826 /// The pid of the run's *foreground* process — the leader of the process
827 /// group the PTY currently gives the keyboard to.
828 ///
829 /// For a shell tile that is the shell itself while it sits at a prompt,
830 /// and the command the operator is running while one is in flight. That
831 /// distinction is the whole point: asking the spawned child would report
832 /// the shell forever, so anything derived from this pid (a live cwd probe,
833 /// a "what is this pane doing" label) would answer for the wrong process.
834 ///
835 /// `None` when the run is not active on this driver instance, or when the
836 /// platform has no notion of a foreground process group.
837 pub fn foreground_pid(&self, id: &TaskRunId) -> Option<u32> {
838 let master = self
839 .active
840 .lock()
841 .unwrap()
842 .get(&id.to_string())
843 .map(|c| Arc::clone(&c.master))?;
844 #[cfg(unix)]
845 {
846 let pid = master.lock().unwrap().process_group_leader()?;
847 u32::try_from(pid).ok()
848 }
849 #[cfg(not(unix))]
850 {
851 let _ = master;
852 None
853 }
854 }
855
856 /// Send `signal` to a running task. Defaults to SIGTERM (15).
857 ///
858 /// For SIGTERM, the driver waits up to 5 seconds for the process to exit
859 /// before escalating to SIGKILL. Returns `DriverError::NotFound` if the
860 /// run is not active (already exited or launched on a different driver
861 /// instance).
862 pub async fn kill_run(&self, id: &TaskRunId, signal: Option<i32>) -> Result<(), DriverError> {
863 let kill_tx = self
864 .active
865 .lock()
866 .unwrap()
867 .get(&id.to_string())
868 .map(|c| c.kill_tx.clone());
869
870 match kill_tx {
871 Some(tx) => tx
872 .send(KillRequest { signal: signal.unwrap_or(SIGTERM) })
873 .await
874 .map_err(|_| DriverError::NotFound(id.to_string())),
875 None => Err(DriverError::NotFound(id.to_string())),
876 }
877 }
878
879 /// Write bytes to the stdin of a running task (requires `stdin_enabled`).
880 pub async fn send_stdin(&self, id: &TaskRunId, bytes: Vec<u8>) -> Result<(), DriverError> {
881 let stdin_tx = self
882 .active
883 .lock()
884 .unwrap()
885 .get(&id.to_string())
886 .and_then(|c| c.stdin_tx.clone());
887
888 match stdin_tx {
889 Some(tx) => tx
890 .send(bytes)
891 .await
892 .map_err(|_| DriverError::NotFound(id.to_string())),
893 None => Err(DriverError::NotFound(id.to_string())),
894 }
895 }
896}
897
898// ─── Log fd receiver ─────────────────────────────────────────────────────────
899
900/// Read JSON-lines from the side-channel FIFO read end and store them as
901/// [`EventSource::Shim`] events.
902///
903/// Runs on a dedicated OS thread; exits when the read end sees EOF. EOF
904/// arrives after both the child process AND the lifecycle task have closed
905/// their write ends of the FIFO. The FIFO file is deleted on exit.
906#[cfg(unix)]
907fn run_log_receiver(
908 rt: tokio::runtime::Handle,
909 store: Arc<TaskStore>,
910 run_id: TaskRunId,
911 read_fd: libc::c_int,
912 fifo_path: std::path::PathBuf,
913 started_at_ms: u64,
914) {
915 use std::io::BufRead;
916 use std::os::unix::io::FromRawFd;
917
918 // SAFETY: `read_fd` is a valid, open FIFO fd handed exclusively to this
919 // thread. `File` takes ownership and closes the fd on drop.
920 let file = unsafe { std::fs::File::from_raw_fd(read_fd) };
921 let reader = std::io::BufReader::new(file);
922
923 for line in reader.lines() {
924 let line = match line {
925 Ok(l) => l,
926 Err(_) => break,
927 };
928 let trimmed = line.trim();
929 if trimmed.is_empty() {
930 continue;
931 }
932 let rec: ShimRecord = match serde_json::from_str(trimmed) {
933 Ok(r) => r,
934 Err(_) => continue, // skip malformed lines silently
935 };
936 let level = rec.level.parse::<crate::types::Level>().unwrap_or(crate::types::Level::Info);
937 let source = crate::types::EventSource::Shim {
938 lib: rec.lib.unwrap_or_else(|| "unknown".to_string()),
939 version: rec.lib_version.unwrap_or_else(|| "0.0.0".to_string()),
940 };
941 let fields = if rec.fields.is_object() {
942 rec.fields
943 } else {
944 serde_json::Value::Object(Default::default())
945 };
946 let offset = elapsed_ms(started_at_ms);
947 let _ = rt.block_on(store.append_event(
948 &run_id,
949 offset,
950 level,
951 &rec.target,
952 &rec.msg,
953 &fields,
954 None,
955 &source,
956 ));
957 }
958
959 // Clean up the FIFO file now that the receiver has drained.
960 let _ = std::fs::remove_file(&fifo_path);
961}
962
963// ─── Lifecycle task ───────────────────────────────────────────────────────────
964
965async fn run_lifecycle(
966 store: Arc<TaskStore>,
967 active: Arc<Mutex<HashMap<String, RunControl>>>,
968 id: TaskRunId,
969 pid: u32,
970 child: Box<dyn portable_pty::Child + Send>,
971 master: Arc<Mutex<Box<dyn portable_pty::MasterPty + Send>>>,
972 mut kill_rx: mpsc::Receiver<KillRequest>,
973 reader_done_rx: oneshot::Receiver<()>,
974 completion_tx: Option<tokio::sync::mpsc::UnboundedSender<(TaskRunId, RunStatus)>>,
975 // Holds the write end of the log FIFO open until this task completes.
976 // Dropping it produces EOF for the receiver thread, which happens after
977 // the terminal RunStatus is written below.
978 #[cfg(unix)]
979 _log_wfd: Option<FdCloser>,
980) {
981 // Pin the reader-done future so it can be polled by reference in
982 // nested select! arms without consuming ownership.
983 let reader_done = async { reader_done_rx.await.ok(); };
984 tokio::pin!(reader_done);
985
986 let sent_signal: Option<i32>;
987
988 tokio::select! {
989 req = kill_rx.recv() => {
990 match req {
991 Some(KillRequest { signal }) => {
992 send_unix_signal(pid, signal);
993 if signal == SIGKILL {
994 sent_signal = Some(SIGKILL);
995 } else {
996 // Grace period: give the process a chance to exit cleanly.
997 tokio::select! {
998 _ = &mut reader_done => {
999 // Exited within grace — no SIGKILL needed.
1000 sent_signal = Some(signal);
1001 }
1002 _ = tokio::time::sleep(DEFAULT_GRACE) => {
1003 // Grace expired — escalate.
1004 send_unix_signal(pid, SIGKILL);
1005 sent_signal = Some(SIGKILL);
1006 }
1007 }
1008 }
1009 }
1010 // kill_tx dropped (driver shutting down) — force kill.
1011 None => {
1012 send_unix_signal(pid, SIGKILL);
1013 sent_signal = Some(SIGKILL);
1014 }
1015 }
1016 }
1017 _ = &mut reader_done => {
1018 sent_signal = None;
1019 }
1020 }
1021
1022 // Reap the child (blocking) on a dedicated thread-pool slot.
1023 // Move our master handle in here so the PTY fd outlives the wait. The
1024 // matching `RunControl` (removed from `active` below) holds the other
1025 // `Arc`, so the fd actually closes once both are gone.
1026 let exit_code = task::spawn_blocking(move || {
1027 let mut c = child;
1028 let _m = master; // dropped after wait() returns
1029 c.wait().ok().map(|s| s.exit_code())
1030 })
1031 .await
1032 .ok()
1033 .flatten();
1034
1035 let ended_at = unix_now_secs();
1036 let status = match sent_signal {
1037 Some(sig) => RunStatus::Killed { signal: sig, ended_at },
1038 None => match exit_code {
1039 Some(code) => RunStatus::Done { exit_code: code as i32, ended_at },
1040 None => RunStatus::Lost {
1041 reason: "process exited without an exit code".to_string(),
1042 },
1043 },
1044 };
1045
1046 /* Losing this write is not cosmetic: the run stays `Running` in the store
1047 forever and every reader — tail loops, the terminal UI, the next
1048 daemon's Lost-on-disappear sweep — believes a dead process is alive.
1049 `update_status` already retries through lock contention, so a failure
1050 here is terminal and worth saying out loud. */
1051 if let Err(e) = store.update_status(&id, &status).await {
1052 eprintln!("[yah task-runs] failed to record terminal status for run {id}: {e}");
1053 }
1054 if let Some(ref tx) = completion_tx {
1055 let _ = tx.send((id.clone(), status));
1056 }
1057 active.lock().unwrap().remove(&id.to_string());
1058}
1059
1060// ─── Helpers ──────────────────────────────────────────────────────────────────
1061
1062fn send_unix_signal(pid: u32, signal: i32) {
1063 #[cfg(unix)]
1064 unsafe {
1065 libc::kill(pid as libc::pid_t, signal);
1066 }
1067 // On non-Unix platforms signal delivery is not implemented here.
1068}
1069
1070fn unix_now_secs() -> u64 {
1071 SystemTime::now()
1072 .duration_since(UNIX_EPOCH)
1073 .unwrap_or_default()
1074 .as_secs()
1075}
1076
1077fn elapsed_ms(started_at_ms: u64) -> u32 {
1078 let now_ms = SystemTime::now()
1079 .duration_since(UNIX_EPOCH)
1080 .unwrap_or_default()
1081 .as_millis() as u64;
1082 now_ms.saturating_sub(started_at_ms).min(u32::MAX as u64) as u32
1083}
1084
1085// ─── Tests ────────────────────────────────────────────────────────────────────
1086
1087#[cfg(test)]
1088mod tests {
1089 use super::*;
1090 use crate::store::ChunkFilter;
1091
1092 async fn open_store(dir: &tempfile::TempDir) -> Arc<TaskStore> {
1093 Arc::new(TaskStore::open(&dir.path().join("tr.turso")).await.unwrap())
1094 }
1095
1096 // ── Lost-on-disappear (pure store, no PTY) ────────────────────────────────
1097
1098 #[tokio::test]
1099 async fn lost_on_disappear_marks_stale_running_runs() {
1100 let dir = tempfile::tempdir().unwrap();
1101 let store = open_store(&dir).await;
1102
1103 // Simulate a run left in "Running" state by a prior daemon.
1104 let stale_id = TaskRunId::new();
1105 store
1106 .insert_run(&TaskRunMeta {
1107 id: stale_id.clone(),
1108 command: "sleep 9999".to_string(),
1109 cwd: "/tmp".into(),
1110 env: vec![],
1111 started_at: unix_now_secs() - 60,
1112 status: RunStatus::Running,
1113 label: None,
1114 initiator: Initiator::Human { camp: "test".to_string() },
1115 beholder_status: None,
1116 pinned: false,
1117 origin: None,
1118 host_pid: None,
1119 })
1120 .await
1121 .unwrap();
1122
1123 // Creating a new driver must mark stale runs Lost.
1124 let _driver = TaskDriver::new(Arc::clone(&store)).await.unwrap();
1125
1126 let meta = store.get_run(&stale_id).await.unwrap().unwrap();
1127 assert!(
1128 matches!(meta.status, RunStatus::Lost { .. }),
1129 "stale run should be Lost, got {:?}",
1130 meta.status
1131 );
1132 }
1133
1134 // ── Stale-run policy (R617-F6) ───────────────────────────────────────────
1135
1136 /// Plant a `Running` row as if some other process had spawned it.
1137 async fn plant_running(
1138 store: &Arc<TaskStore>,
1139 origin: Option<&str>,
1140 host_pid: Option<u32>,
1141 ) -> TaskRunId {
1142 let id = TaskRunId::new();
1143 store
1144 .insert_run(&TaskRunMeta {
1145 id: id.clone(),
1146 command: "sleep 9999".to_string(),
1147 cwd: "/tmp".into(),
1148 env: vec![],
1149 started_at: unix_now_secs() - 60,
1150 status: RunStatus::Running,
1151 label: None,
1152 initiator: Initiator::Human {
1153 camp: "test".to_string(),
1154 },
1155 beholder_status: None,
1156 pinned: false,
1157 origin: origin.map(str::to_string),
1158 host_pid,
1159 })
1160 .await
1161 .unwrap();
1162 id
1163 }
1164
1165 async fn is_lost(store: &Arc<TaskStore>, id: &TaskRunId) -> bool {
1166 matches!(
1167 store.get_run(id).await.unwrap().unwrap().status,
1168 RunStatus::Lost { .. }
1169 )
1170 }
1171
1172 fn adopt_terminal() -> StaleRunPolicy {
1173 StaleRunPolicy::AdoptLiveHosts {
1174 origins: vec!["terminal".to_string()],
1175 }
1176 }
1177
1178 /// The property the whole ticket exists for: attaching to a store must not
1179 /// declare another live process's shell dead.
1180 #[tokio::test]
1181 async fn a_run_owned_by_a_live_host_survives_a_new_driver() {
1182 let dir = tempfile::tempdir().unwrap();
1183 let store = open_store(&dir).await;
1184 // Our own pid is by definition a live process, and is the cheapest
1185 // honest stand-in for "a peer that is still running".
1186 let id = plant_running(&store, Some("terminal"), Some(std::process::id())).await;
1187
1188 let _driver = TaskDriver::with_config(
1189 Arc::clone(&store),
1190 DriverChannels::default(),
1191 adopt_terminal(),
1192 )
1193 .await
1194 .unwrap();
1195
1196 assert!(
1197 !is_lost(&store, &id).await,
1198 "a terminal run whose owner is alive must stay Running — \
1199 tombstoning it is what made a surviving shell read as dead"
1200 );
1201 }
1202
1203 /// The other half: a genuinely abandoned shell must still be tombstoned,
1204 /// or a crashed host leaves permanent zombie tiles.
1205 #[tokio::test]
1206 async fn a_run_whose_host_is_gone_is_still_tombstoned() {
1207 let dir = tempfile::tempdir().unwrap();
1208 let store = open_store(&dir).await;
1209 // Reaped in-test, so the pid is real-but-dead rather than guessed.
1210 let dead_pid = {
1211 let child = std::process::Command::new("true").spawn().unwrap();
1212 let pid = child.id();
1213 let mut child = child;
1214 let _ = child.wait();
1215 pid
1216 };
1217 let id = plant_running(&store, Some("terminal"), Some(dead_pid)).await;
1218
1219 let _driver = TaskDriver::with_config(
1220 Arc::clone(&store),
1221 DriverChannels::default(),
1222 adopt_terminal(),
1223 )
1224 .await
1225 .unwrap();
1226
1227 assert!(
1228 is_lost(&store, &id).await,
1229 "pid {dead_pid} was reaped; its run has no owner left and must be Lost"
1230 );
1231 }
1232
1233 /// The exemption is narrowed by origin, so ordinary jobs keep the old rule
1234 /// even when their owner happens to still be alive — an in-flight `cargo
1235 /// build` whose driver is gone has nobody left to record its exit.
1236 #[tokio::test]
1237 async fn a_non_matching_origin_is_tombstoned_even_with_a_live_host() {
1238 let dir = tempfile::tempdir().unwrap();
1239 let store = open_store(&dir).await;
1240 let job = plant_running(&store, None, Some(std::process::id())).await;
1241 let other = plant_running(&store, Some("gnome"), Some(std::process::id())).await;
1242
1243 let _driver = TaskDriver::with_config(
1244 Arc::clone(&store),
1245 DriverChannels::default(),
1246 adopt_terminal(),
1247 )
1248 .await
1249 .unwrap();
1250
1251 assert!(is_lost(&store, &job).await, "an origin-less job is not exempt");
1252 assert!(
1253 is_lost(&store, &other).await,
1254 "an origin outside the list is not exempt"
1255 );
1256 }
1257
1258 /// A row written before `host_pid` existed reads back `None`. Unknown
1259 /// ownership must fall back to the old behaviour rather than stranding the
1260 /// run `Running` forever.
1261 #[tokio::test]
1262 async fn an_unattributed_run_is_tombstoned() {
1263 let dir = tempfile::tempdir().unwrap();
1264 let store = open_store(&dir).await;
1265 let id = plant_running(&store, Some("terminal"), None).await;
1266
1267 let _driver = TaskDriver::with_config(
1268 Arc::clone(&store),
1269 DriverChannels::default(),
1270 adopt_terminal(),
1271 )
1272 .await
1273 .unwrap();
1274
1275 assert!(is_lost(&store, &id).await);
1276 }
1277
1278 /// `TaskDriver::new` must not have quietly changed behaviour — every
1279 /// existing embedder still gets Lost-on-disappear.
1280 #[tokio::test]
1281 async fn the_default_policy_is_still_lost_on_disappear() {
1282 assert_eq!(StaleRunPolicy::default(), StaleRunPolicy::LostOnDisappear);
1283
1284 let dir = tempfile::tempdir().unwrap();
1285 let store = open_store(&dir).await;
1286 let id = plant_running(&store, Some("terminal"), Some(std::process::id())).await;
1287
1288 let _driver = TaskDriver::new(Arc::clone(&store)).await.unwrap();
1289
1290 assert!(
1291 is_lost(&store, &id).await,
1292 "the default must tombstone regardless of origin or owner liveness"
1293 );
1294 }
1295
1296 /// The owner is recorded by `spawn_run` itself, not by the caller — the
1297 /// policy is worthless if rows arrive unattributed.
1298 #[tokio::test]
1299 async fn spawn_run_stamps_this_process_as_the_owner() {
1300 let dir = tempfile::tempdir().unwrap();
1301 let store = open_store(&dir).await;
1302 let driver = TaskDriver::new(Arc::clone(&store)).await.unwrap();
1303
1304 let id = driver
1305 .spawn_run(
1306 "true",
1307 SpawnOpts {
1308 cwd: "/tmp".into(),
1309 origin: Some("terminal".to_string()),
1310 ..Default::default()
1311 },
1312 )
1313 .await
1314 .unwrap();
1315
1316 let meta = store.get_run(&id).await.unwrap().unwrap();
1317 assert_eq!(meta.host_pid, Some(std::process::id()));
1318 }
1319
1320 #[tokio::test]
1321 async fn new_driver_does_not_touch_completed_runs() {
1322 let dir = tempfile::tempdir().unwrap();
1323 let store = open_store(&dir).await;
1324
1325 let done_id = TaskRunId::new();
1326 store
1327 .insert_run(&TaskRunMeta {
1328 id: done_id.clone(),
1329 command: "true".to_string(),
1330 cwd: "/tmp".into(),
1331 env: vec![],
1332 started_at: unix_now_secs() - 10,
1333 status: RunStatus::Running,
1334 label: None,
1335 initiator: Initiator::Human { camp: "test".to_string() },
1336 beholder_status: None,
1337 pinned: false,
1338 origin: None,
1339 host_pid: None,
1340 })
1341 .await
1342 .unwrap();
1343 store
1344 .update_status(&done_id, &RunStatus::Done { exit_code: 0, ended_at: unix_now_secs() })
1345 .await
1346 .unwrap();
1347
1348 let _driver = TaskDriver::new(Arc::clone(&store)).await.unwrap();
1349
1350 let meta = store.get_run(&done_id).await.unwrap().unwrap();
1351 assert!(
1352 matches!(meta.status, RunStatus::Done { .. }),
1353 "completed run must not be touched"
1354 );
1355 }
1356
1357 // ── PTY spawn + capture ───────────────────────────────────────────────────
1358
1359 #[tokio::test]
1360 async fn spawn_echo_and_read_chunks() {
1361 let dir = tempfile::tempdir().unwrap();
1362 let store = open_store(&dir).await;
1363 let driver = TaskDriver::new(Arc::clone(&store)).await.unwrap();
1364
1365 let id = driver
1366 .spawn_run(
1367 "echo hello_world",
1368 SpawnOpts { cwd: "/tmp".into(), ..Default::default() },
1369 )
1370 .await
1371 .unwrap();
1372
1373 // Wait for the run to complete (poll status up to 5 s).
1374 let deadline = std::time::Instant::now() + Duration::from_secs(5);
1375 loop {
1376 let meta = store.get_run(&id).await.unwrap().unwrap();
1377 if matches!(meta.status, RunStatus::Done { .. } | RunStatus::Lost { .. }) {
1378 break;
1379 }
1380 if std::time::Instant::now() > deadline {
1381 panic!("run did not complete in time, status={:?}", meta.status);
1382 }
1383 tokio::time::sleep(Duration::from_millis(50)).await;
1384 }
1385
1386 // Chunks must contain "hello_world".
1387 let chunks = store
1388 .get_chunks(&id, &ChunkFilter::default())
1389 .await
1390 .unwrap();
1391 let output: Vec<u8> = chunks.into_iter().flat_map(|c| c.bytes).collect();
1392 let text = String::from_utf8_lossy(&output);
1393 assert!(
1394 text.contains("hello_world"),
1395 "expected 'hello_world' in output, got: {text:?}"
1396 );
1397
1398 let meta = store.get_run(&id).await.unwrap().unwrap();
1399 assert!(
1400 matches!(meta.status, RunStatus::Done { exit_code: 0, .. }),
1401 "expected Done(0), got {:?}",
1402 meta.status
1403 );
1404 }
1405
1406 /// Wait for a run to reach a terminal status, or panic.
1407 async fn await_done(store: &TaskStore, id: &TaskRunId) -> TaskRunMeta {
1408 let deadline = std::time::Instant::now() + Duration::from_secs(5);
1409 loop {
1410 let meta = store.get_run(id).await.unwrap().unwrap();
1411 if matches!(meta.status, RunStatus::Done { .. } | RunStatus::Lost { .. }) {
1412 return meta;
1413 }
1414 if std::time::Instant::now() > deadline {
1415 panic!("run did not complete in time, status={:?}", meta.status);
1416 }
1417 tokio::time::sleep(Duration::from_millis(50)).await;
1418 }
1419 }
1420
1421 async fn output_of(store: &TaskStore, id: &TaskRunId) -> String {
1422 let chunks = store.get_chunks(id, &ChunkFilter::default()).await.unwrap();
1423 let bytes: Vec<u8> = chunks.into_iter().flat_map(|c| c.bytes).collect();
1424 String::from_utf8_lossy(&bytes).into_owned()
1425 }
1426
1427 // ── Direct argv (R652-T6) ────────────────────────────────────────────────
1428
1429 #[tokio::test]
1430 async fn explicit_argv_execs_the_program_directly() {
1431 let dir = tempfile::tempdir().unwrap();
1432 let store = open_store(&dir).await;
1433 let driver = TaskDriver::new(Arc::clone(&store)).await.unwrap();
1434
1435 /* The distinguishing observation: under `sh -c` the child is `sh` and
1436 `$0` is `sh`; exec'd directly it is the program itself. Printing
1437 `$0` is the cheapest way to see which of the two happened. */
1438 let id = driver
1439 .spawn_run(
1440 "unused-because-argv-wins",
1441 SpawnOpts {
1442 cwd: "/tmp".into(),
1443 argv: Some(vec![
1444 "/bin/sh".into(),
1445 "-c".into(),
1446 "printf 'argv0=%s\\n' \"$0\"".into(),
1447 "direct-exec-marker".into(),
1448 ]),
1449 ..Default::default()
1450 },
1451 )
1452 .await
1453 .unwrap();
1454
1455 await_done(&store, &id).await;
1456 let text = output_of(&store, &id).await;
1457 assert!(
1458 text.contains("argv0=direct-exec-marker"),
1459 "argv should have been exec'd verbatim, got: {text:?}"
1460 );
1461 }
1462
1463 #[tokio::test]
1464 async fn explicit_argv_still_records_the_requested_command() {
1465 let dir = tempfile::tempdir().unwrap();
1466 let store = open_store(&dir).await;
1467 let driver = TaskDriver::new(Arc::clone(&store)).await.unwrap();
1468
1469 /* A shell tile asks for "$SHELL" and the daemon resolves it to a real
1470 argv. The run must still read back as what was asked for, or the
1471 rail row and the history re-run both show an implementation
1472 detail. */
1473 let id = driver
1474 .spawn_run(
1475 "$SHELL",
1476 SpawnOpts {
1477 cwd: "/tmp".into(),
1478 argv: Some(vec!["/bin/sh".into(), "-c".into(), "true".into()]),
1479 ..Default::default()
1480 },
1481 )
1482 .await
1483 .unwrap();
1484
1485 let meta = await_done(&store, &id).await;
1486 assert_eq!(meta.command, "$SHELL");
1487 assert!(
1488 matches!(meta.status, RunStatus::Done { exit_code: 0, .. }),
1489 "expected Done(0), got {:?}",
1490 meta.status
1491 );
1492 }
1493
1494 #[tokio::test]
1495 async fn empty_argv_falls_back_to_the_shell_path() {
1496 let dir = tempfile::tempdir().unwrap();
1497 let store = open_store(&dir).await;
1498 let driver = TaskDriver::new(Arc::clone(&store)).await.unwrap();
1499
1500 let id = driver
1501 .spawn_run(
1502 "echo empty_argv_fallback",
1503 SpawnOpts { cwd: "/tmp".into(), argv: Some(vec![]), ..Default::default() },
1504 )
1505 .await
1506 .unwrap();
1507
1508 await_done(&store, &id).await;
1509 let text = output_of(&store, &id).await;
1510 assert!(
1511 text.contains("empty_argv_fallback"),
1512 "empty argv must not spawn nothing, got: {text:?}"
1513 );
1514 }
1515
1516 #[tokio::test]
1517 async fn spawn_failing_command_records_nonzero_exit() {
1518 let dir = tempfile::tempdir().unwrap();
1519 let store = open_store(&dir).await;
1520 let driver = TaskDriver::new(Arc::clone(&store)).await.unwrap();
1521
1522 let id = driver
1523 .spawn_run(
1524 "exit 42",
1525 SpawnOpts { cwd: "/tmp".into(), ..Default::default() },
1526 )
1527 .await
1528 .unwrap();
1529
1530 let deadline = std::time::Instant::now() + Duration::from_secs(5);
1531 loop {
1532 let meta = store.get_run(&id).await.unwrap().unwrap();
1533 if !matches!(meta.status, RunStatus::Running | RunStatus::Pending) {
1534 match meta.status {
1535 RunStatus::Done { exit_code, .. } => {
1536 assert_ne!(exit_code, 0, "exit 42 should produce a non-zero exit code");
1537 }
1538 other => panic!("unexpected status: {other:?}"),
1539 }
1540 break;
1541 }
1542 if std::time::Instant::now() > deadline {
1543 panic!("run did not complete in time");
1544 }
1545 tokio::time::sleep(Duration::from_millis(50)).await;
1546 }
1547 }
1548
1549 // ── Signal handling ───────────────────────────────────────────────────────
1550
1551 #[cfg(unix)]
1552 #[tokio::test]
1553 async fn kill_with_sigterm_transitions_to_killed() {
1554 let dir = tempfile::tempdir().unwrap();
1555 let store = open_store(&dir).await;
1556 let driver = Arc::new(TaskDriver::new(Arc::clone(&store)).await.unwrap());
1557
1558 let id = driver
1559 .spawn_run(
1560 "sleep 60",
1561 SpawnOpts { cwd: "/tmp".into(), ..Default::default() },
1562 )
1563 .await
1564 .unwrap();
1565
1566 // Give the process a moment to start.
1567 tokio::time::sleep(Duration::from_millis(100)).await;
1568
1569 driver.kill_run(&id, Some(SIGTERM)).await.unwrap();
1570
1571 let deadline = std::time::Instant::now() + Duration::from_secs(10);
1572 loop {
1573 let meta = store.get_run(&id).await.unwrap().unwrap();
1574 if matches!(meta.status, RunStatus::Killed { .. } | RunStatus::Lost { .. }) {
1575 assert!(
1576 matches!(meta.status, RunStatus::Killed { .. }),
1577 "expected Killed, got {:?}",
1578 meta.status
1579 );
1580 break;
1581 }
1582 if std::time::Instant::now() > deadline {
1583 panic!("run did not become Killed in time, status={:?}", meta.status);
1584 }
1585 tokio::time::sleep(Duration::from_millis(50)).await;
1586 }
1587 }
1588
1589 #[cfg(unix)]
1590 #[tokio::test]
1591 async fn kill_run_returns_not_found_after_exit() {
1592 let dir = tempfile::tempdir().unwrap();
1593 let store = open_store(&dir).await;
1594 let driver = Arc::new(TaskDriver::new(Arc::clone(&store)).await.unwrap());
1595
1596 let id = driver
1597 .spawn_run(
1598 "echo done",
1599 SpawnOpts { cwd: "/tmp".into(), ..Default::default() },
1600 )
1601 .await
1602 .unwrap();
1603
1604 // Wait for natural exit.
1605 let deadline = std::time::Instant::now() + Duration::from_secs(5);
1606 loop {
1607 let meta = store.get_run(&id).await.unwrap().unwrap();
1608 if !matches!(meta.status, RunStatus::Running | RunStatus::Pending) {
1609 break;
1610 }
1611 if std::time::Instant::now() > deadline {
1612 panic!("run did not complete");
1613 }
1614 tokio::time::sleep(Duration::from_millis(50)).await;
1615 }
1616
1617 // Kill on a completed run should return NotFound.
1618 let result = driver.kill_run(&id, None).await;
1619 assert!(
1620 matches!(result, Err(DriverError::NotFound(_))),
1621 "expected NotFound, got {result:?}"
1622 );
1623 }
1624
1625 // ── Stdin relay ───────────────────────────────────────────────────────────
1626
1627 #[cfg(unix)]
1628 #[tokio::test]
1629 async fn stdin_send_reaches_child() {
1630 let dir = tempfile::tempdir().unwrap();
1631 let store = open_store(&dir).await;
1632 let driver = Arc::new(TaskDriver::new(Arc::clone(&store)).await.unwrap());
1633
1634 // Shell that reads a line from stdin and echoes it back.
1635 let id = driver
1636 .spawn_run(
1637 "read line && echo got_$line",
1638 SpawnOpts {
1639 cwd: "/tmp".into(),
1640 stdin_enabled: true,
1641 ..Default::default()
1642 },
1643 )
1644 .await
1645 .unwrap();
1646
1647 tokio::time::sleep(Duration::from_millis(150)).await;
1648 driver.send_stdin(&id, b"hello\n".to_vec()).await.unwrap();
1649
1650 let deadline = std::time::Instant::now() + Duration::from_secs(5);
1651 loop {
1652 let meta = store.get_run(&id).await.unwrap().unwrap();
1653 if !matches!(meta.status, RunStatus::Running | RunStatus::Pending) {
1654 break;
1655 }
1656 if std::time::Instant::now() > deadline {
1657 panic!("run did not complete after stdin input");
1658 }
1659 tokio::time::sleep(Duration::from_millis(50)).await;
1660 }
1661
1662 let chunks = store.get_chunks(&id, &ChunkFilter::default()).await.unwrap();
1663 let raw: Vec<u8> = chunks.into_iter().flat_map(|c| c.bytes).collect();
1664 let text = String::from_utf8_lossy(&raw);
1665 assert!(
1666 text.contains("got_hello"),
1667 "expected 'got_hello' in output, got: {text:?}"
1668 );
1669 }
1670
1671 /// `resize_run` must change the geometry the *child* sees, not just the
1672 /// master fd — so the assertion reads `stty size` from inside the PTY
1673 /// after the resize rather than inspecting the driver's own state.
1674 #[tokio::test]
1675 async fn resize_run_changes_geometry_the_child_sees() {
1676 let dir = tempfile::tempdir().unwrap();
1677 let store = open_store(&dir).await;
1678 let driver = Arc::new(TaskDriver::new(Arc::clone(&store)).await.unwrap());
1679
1680 // Wait for a line on stdin, then report the geometry as of that moment.
1681 let id = driver
1682 .spawn_run(
1683 "read line && stty size",
1684 SpawnOpts {
1685 cwd: "/tmp".into(),
1686 stdin_enabled: true,
1687 // Spawn at the default 80x24 so the assertion can't pass by
1688 // accident if the resize is a no-op.
1689 ..Default::default()
1690 },
1691 )
1692 .await
1693 .unwrap();
1694
1695 tokio::time::sleep(Duration::from_millis(150)).await;
1696 driver.resize_run(&id, 120, 40).await.unwrap();
1697 driver.send_stdin(&id, b"go\n".to_vec()).await.unwrap();
1698
1699 let deadline = std::time::Instant::now() + Duration::from_secs(5);
1700 loop {
1701 let meta = store.get_run(&id).await.unwrap().unwrap();
1702 if !matches!(meta.status, RunStatus::Running | RunStatus::Pending) {
1703 break;
1704 }
1705 if std::time::Instant::now() > deadline {
1706 panic!("run did not complete after stdin input");
1707 }
1708 tokio::time::sleep(Duration::from_millis(50)).await;
1709 }
1710
1711 let chunks = store.get_chunks(&id, &ChunkFilter::default()).await.unwrap();
1712 let raw: Vec<u8> = chunks.into_iter().flat_map(|c| c.bytes).collect();
1713 let text = String::from_utf8_lossy(&raw);
1714 assert!(
1715 text.contains("40 120"),
1716 "expected resized geometry '40 120' in output, got: {text:?}"
1717 );
1718 }
1719
1720 /// A run that is not active on this driver (finished, or never existed) is
1721 /// `NotFound` rather than a panic — same contract as `send_stdin`.
1722 #[tokio::test]
1723 async fn resize_run_returns_not_found_after_exit() {
1724 let dir = tempfile::tempdir().unwrap();
1725 let store = open_store(&dir).await;
1726 let driver = Arc::new(TaskDriver::new(Arc::clone(&store)).await.unwrap());
1727
1728 let id = driver
1729 .spawn_run("true", SpawnOpts { cwd: "/tmp".into(), ..Default::default() })
1730 .await
1731 .unwrap();
1732
1733 let deadline = std::time::Instant::now() + Duration::from_secs(5);
1734 loop {
1735 let meta = store.get_run(&id).await.unwrap().unwrap();
1736 if !matches!(meta.status, RunStatus::Running | RunStatus::Pending) {
1737 break;
1738 }
1739 if std::time::Instant::now() > deadline {
1740 panic!("run did not exit");
1741 }
1742 tokio::time::sleep(Duration::from_millis(50)).await;
1743 }
1744
1745 assert!(matches!(
1746 driver.resize_run(&id, 100, 30).await,
1747 Err(DriverError::NotFound(_))
1748 ));
1749 }
1750
1751 // ── Tier-2 side-channel log fd ────────────────────────────────────────────
1752
1753 /// Verify that a child writing a JSON-line to `YAH_LOG_PIPE` (via
1754 /// `printf ... >> $YAH_LOG_PIPE`) produces a shim event with the correct
1755 /// fields in the store.
1756 ///
1757 /// The child opens the FIFO path for writing — no fd inheritance needed.
1758 #[cfg(unix)]
1759 #[tokio::test(flavor = "multi_thread", worker_threads = 4)]
1760 async fn log_pipe_events_land_in_store() {
1761 use crate::store::EventFilter;
1762
1763 let dir = tempfile::tempdir().unwrap();
1764 let store = open_store(&dir).await;
1765 let driver = Arc::new(TaskDriver::new(Arc::clone(&store)).await.unwrap());
1766
1767 // The shell writes one JSON-line to the FIFO by redirecting printf
1768 // output to the path stored in YAH_LOG_PIPE.
1769 let cmd = r#"printf '{"level":"warn","target":"test.shim","msg":"hello-from-pipe","fields":{"x":42},"_lib":"test-shim","_lib_ver":"0.1.0"}\n' >> "$YAH_LOG_PIPE""#;
1770
1771 let id = driver
1772 .spawn_run(cmd, SpawnOpts { cwd: "/tmp".into(), ..Default::default() })
1773 .await
1774 .unwrap();
1775
1776 // Wait for run completion. Deadline is generous because parallel-test
1777 // load + the rt.block_on hops from the reader/log threads can slow
1778 // child-process scheduling.
1779 let deadline = std::time::Instant::now() + Duration::from_secs(20);
1780 loop {
1781 let meta = store.get_run(&id).await.unwrap().unwrap();
1782 if matches!(meta.status, RunStatus::Done { .. } | RunStatus::Lost { .. }) {
1783 break;
1784 }
1785 if std::time::Instant::now() > deadline {
1786 panic!("run did not complete in time");
1787 }
1788 tokio::time::sleep(Duration::from_millis(50)).await;
1789 }
1790
1791 // The log receiver thread drains after the lifecycle task drops the
1792 // write-end FdCloser; give it a brief moment.
1793 tokio::time::sleep(Duration::from_millis(500)).await;
1794
1795 let events = store.query_events(&id, &EventFilter::default()).await.unwrap();
1796 assert!(
1797 !events.is_empty(),
1798 "expected at least one shim event, got none"
1799 );
1800 let ev = events.iter().find(|e| e.target == "test.shim");
1801 let ev = ev.expect("event with target 'test.shim' not found");
1802 assert_eq!(ev.msg, "hello-from-pipe");
1803 assert_eq!(ev.level, crate::types::Level::Warn);
1804 assert!(
1805 matches!(&ev.source, crate::types::EventSource::Shim { lib, .. } if lib == "test-shim"),
1806 "unexpected source: {:?}",
1807 ev.source
1808 );
1809 assert_eq!(ev.fields.get("x"), Some(&serde_json::json!(42)));
1810 }
1811
1812 /// When `log_fd_enabled` is false, neither `YAH_TASK_RUN` nor
1813 /// `YAH_LOG_PIPE` are exported, and no shim events are written.
1814 #[cfg(unix)]
1815 #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
1816 async fn log_pipe_disabled_produces_no_events() {
1817 use crate::store::EventFilter;
1818
1819 let dir = tempfile::tempdir().unwrap();
1820 let store = open_store(&dir).await;
1821 let driver = Arc::new(TaskDriver::new(Arc::clone(&store)).await.unwrap());
1822
1823 // Try to write to YAH_LOG_PIPE; the conditional guards against
1824 // the variable being absent, so the command always exits 0.
1825 let cmd = r#"[ -n "$YAH_LOG_PIPE" ] && printf '{"level":"info","target":"t","msg":"m","fields":{}}\n' >> "$YAH_LOG_PIPE" || true"#;
1826
1827 let id = driver
1828 .spawn_run(
1829 cmd,
1830 SpawnOpts { cwd: "/tmp".into(), log_fd_enabled: false, ..Default::default() },
1831 )
1832 .await
1833 .unwrap();
1834
1835 let deadline = std::time::Instant::now() + Duration::from_secs(5);
1836 loop {
1837 let meta = store.get_run(&id).await.unwrap().unwrap();
1838 if matches!(meta.status, RunStatus::Done { .. } | RunStatus::Lost { .. }) {
1839 break;
1840 }
1841 if std::time::Instant::now() > deadline {
1842 panic!("run did not complete");
1843 }
1844 tokio::time::sleep(Duration::from_millis(50)).await;
1845 }
1846
1847 tokio::time::sleep(Duration::from_millis(100)).await;
1848
1849 let events = store.query_events(&id, &EventFilter::default()).await.unwrap();
1850 assert!(
1851 events.is_empty(),
1852 "expected no shim events when log_fd_enabled=false, got {}",
1853 events.len()
1854 );
1855 }
1856}