graphrt-core 0.2.1

Executor-agnostic graph runtime for agentic AI workflows: graph model, variable pool, scheduler, events, snapshots and built-in control nodes.
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
//! The execution engine.
//!
//! # Model
//!
//! - The [`Graph`] is immutable and shared; each run keeps its own per-frame
//!   node and edge state.
//! - A node becomes ready once **every** incoming edge is resolved and **at
//!   least one** is taken. A node whose incoming edges are all skipped is
//!   skipped too (`NodeSkipped`), and the skip propagates downstream.
//! - Non-branch nodes take every outgoing edge except `fail-branch` edges.
//!   Branch nodes take the edges whose handle equals [`NodeOutput::branch`];
//!   a branch node that selects no handle fails (subject to its error strategy).
//!   A selected handle with no edges simply skips every outgoing path.
//! - Response nodes (answer / end) merge their outputs into the frame outputs:
//!   `answer` is concatenated, other keys overwrite. The root frame outputs are
//!   the run outputs.
//! - Nodes of one frame run concurrently (up to [`Limits::max_concurrency`]) as
//!   futures polled by the run itself. Nothing is spawned, so the engine runs on
//!   any executor, including single-threaded and wasm ones.
//!
//! # Run bootstrap
//!
//! The root pool receives `env.*` (document defaults coerced to their declared
//! types, overridden by [`RunInput::env`]), `conversation.*` (same rule; missing
//! values get their type's zero value), `sys.*` from [`RunInput::sys`] plus
//! `sys.run_id`. [`RunInput::inputs`] are written into the **namespace of the
//! root entry node** (normally the `start` node) before it runs; the start
//! node validates them and re-emits them as its outputs. Inputs are never
//! copied into `sys`, so an input cannot shadow a system variable.
//!
//! # Failures
//!
//! - **Retries** apply to every node kind when `retry.max_retries > 0` and the
//!   error is [`NodeError::retryable`]. The delay is the larger of the policy
//!   delay and [`NodeError::retry_after_ms`] (capped at five minutes), awaited
//!   inside the node's own future, so other nodes keep running. The execution id is stable across
//!   attempts, and a retry that eventually succeeds is not a handled error.
//! - **Timeouts** (`timeout_ms`, else [`Limits::default_node_timeout_ms`]) apply
//!   per attempt when the clock supports timers. The attempt deadline is also
//!   exposed as [`NodeContext::deadline_ms`](crate::node::NodeContext) so nodes
//!   that block synchronously can bound themselves.
//! - After retries, the error strategy applies:
//!   - `fail`: the frame fails. In the root frame the run ends with `RunFailed`
//!     and in-flight nodes are dropped. In a container body, the container node
//!     receives [`SubgraphResult::Failed`](crate::node::SubgraphResult).
//!   - `fail-branch`: outputs `error_message` / `error_type`; only `fail-branch`
//!     edges are taken.
//!   - `default-value`: outputs are the defaults plus `error_message` /
//!     `error_type`; non-branch nodes then continue as on success. Branch nodes
//!     follow the handle named by a `branch` default value, or skip every
//!     outgoing path if there is none.
//!
//!   Handled failures are counted in `RunSucceeded::handled_errors`.
//! - A panicking node fails with error type `panic` (when unwinding is enabled).
//!
//! # Limits
//!
//! `max_steps` counts every node attempt in every frame and persists across
//! pause / resume. `max_duration_ms` covers the whole run (including time before
//! a pause) and is enforced by a timer race. `max_depth` bounds container nesting.
//!
//! # Containers
//!
//! Container nodes receive [`NodeContext::subgraph`](crate::node::NodeContext),
//! which runs the container's body as a child frame (`"<parent>/<container>[i]"`)
//! with the same scheduler, event sink, step budget and control signals. Writes
//! to namespaces outside the body (container variables, conversation variables)
//! are returned in `SubgraphResult::Completed::writes`; `broke` reports whether a
//! `loop-end` node completed. Usage of container nodes themselves is not added
//! to the run total (their children are already counted).
//!
//! # Pause / resume / abort
//!
//! - [`RunHandle::pause`] stops scheduling everywhere, lets in-flight nodes
//!   finish and then ends with `RunPaused` carrying a [`Snapshot`].
//! - A node returning [`NodeOutcome::Paused`](crate::node::NodeOutcome) pauses
//!   only itself: independent branches keep running until nothing else can make
//!   progress, then the run pauses.
//! - On resume ([`RunInput::resume`]) the snapshot's graph fingerprint and
//!   format are verified, env / sys / conversation values from the new
//!   [`RunInput`] are applied, pending nodes re-run and paused nodes are resumed
//!   with their saved state plus their entry in [`ResumeFrom::inputs`].
//!   Inputs are looked up by the frame-qualified key `"<frame>/<node_id>"`
//!   first (e.g. `"review[2]/approve"` for a node paused inside the third
//!   item of iteration `review`; the frame is [`PauseReason::frame`]), then by
//!   the plain node id, which applies to every frame.
//! - [`RunHandle::abort`] drops every in-flight node future immediately.

mod control;
mod frame;
mod ids;
mod root;

use std::collections::VecDeque;
use std::pin::Pin;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex, PoisonError};
use std::task::{Context, Poll};

use futures::future::BoxFuture;
use futures::task::AtomicWaker;
use futures::{FutureExt, Stream, StreamExt};
use indexmap::IndexMap;

use self::control::{Control, Stop};
use crate::event::{Event, PauseReason};
use crate::graph::Graph;
use crate::node::{Clock, EventSink, Services, default_clock};
use crate::pool::Selector;
use crate::snapshot::Snapshot;
use crate::value::Value;

#[cfg(doc)]
use crate::node::{NodeError, NodeOutput};

/// Resource limits for a run.
#[derive(Clone, Debug, PartialEq)]
pub struct Limits {
    /// Maximum node executions (including retries and container bodies). Default 10 000.
    pub max_steps: u64,
    /// Wall-clock budget for the whole run. Enforced only with a timer-capable clock.
    pub max_duration_ms: Option<u64>,
    /// Maximum container nesting depth. Default 8.
    pub max_depth: u32,
    /// Timeout applied to nodes without their own `timeout_ms`.
    pub default_node_timeout_ms: Option<u64>,
    /// Maximum nodes executing concurrently within one frame. Default 64.
    pub max_concurrency: usize,
}

impl Default for Limits {
    fn default() -> Self {
        Self {
            max_steps: 10_000,
            max_duration_ms: None,
            max_depth: 8,
            default_node_timeout_ms: None,
            max_concurrency: 64,
        }
    }
}

/// Input for a run.
#[derive(Clone, Debug, Default)]
pub struct RunInput {
    /// Values for the start node's declared variables (written into the entry node's namespace).
    pub inputs: IndexMap<String, Value>,
    /// System variables (`sys.*`), e.g. `query`, `files`, `user_id`, `conversation_id`.
    pub sys: IndexMap<String, Value>,
    /// Environment variables (`env.*`); override document defaults. Never persisted.
    pub env: IndexMap<String, Value>,
    /// Conversation variables (`conversation.*`); override document defaults.
    pub conversation: IndexMap<String, Value>,
    /// Explicit run id (generated when absent).
    pub run_id: Option<String>,
    /// Resume a paused run.
    pub resume: Option<ResumeFrom>,
}

impl RunInput {
    pub fn new() -> Self {
        Self::default()
    }

    /// Add a start input.
    pub fn input(mut self, name: impl Into<String>, value: impl Into<Value>) -> Self {
        self.inputs.insert(name.into(), value.into());
        self
    }

    /// Add a system variable.
    pub fn sys(mut self, name: impl Into<String>, value: impl Into<Value>) -> Self {
        self.sys.insert(name.into(), value.into());
        self
    }

    /// Add an environment variable.
    pub fn env(mut self, name: impl Into<String>, value: impl Into<Value>) -> Self {
        self.env.insert(name.into(), value.into());
        self
    }

    /// Resume from a snapshot.
    pub fn resume(mut self, snapshot: Snapshot, inputs: IndexMap<String, Value>) -> Self {
        self.resume = Some(ResumeFrom::new(snapshot, inputs));
        self
    }
}

/// Resume information.
#[derive(Clone, Debug)]
pub struct ResumeFrom {
    pub snapshot: Snapshot,
    /// Host input for paused nodes (e.g. a submitted human-input form), keyed by
    /// `"<frame>/<node_id>"` for one specific frame or by plain node id for any
    /// frame. Root-frame nodes use the plain node id.
    pub inputs: IndexMap<String, Value>,
    /// Plaintext for secrets outside `env.*` (e.g. secret-typed start inputs or
    /// conversation variables), keyed by dotted selector (`"start.api_key"`).
    /// Snapshots never store secret plaintext: such values are restored as
    /// redacted secrets ([`Value::redacted_secret`]) and nodes that need them fail
    /// until they are re-supplied here. Only selectors that currently hold a
    /// redacted secret are overwritten.
    pub secrets: IndexMap<String, Value>,
}

impl ResumeFrom {
    /// Resume `snapshot` with host input for paused nodes.
    pub fn new(snapshot: Snapshot, inputs: IndexMap<String, Value>) -> Self {
        Self { snapshot, inputs, secrets: IndexMap::new() }
    }

    /// Re-supply a secret that is not an environment variable.
    pub fn secret(mut self, selector: impl Into<String>, value: impl Into<String>) -> Self {
        self.secrets.insert(selector.into(), Value::Secret(Arc::from(value.into())));
        self
    }
}

/// Executes runs of one graph.
#[derive(Clone, Debug)]
pub struct Engine {
    graph: Arc<Graph>,
    services: Arc<Services>,
    clock: Arc<dyn Clock>,
    limits: Limits,
}

impl Engine {
    pub fn new(graph: Arc<Graph>) -> Self {
        Self { graph, services: Arc::new(Services::new()), clock: default_clock(), limits: Limits::default() }
    }

    pub fn with_services(mut self, services: Services) -> Self {
        self.services = Arc::new(services);
        self
    }

    /// Share an existing services registry.
    pub fn with_shared_services(mut self, services: Arc<Services>) -> Self {
        self.services = services;
        self
    }

    pub fn with_clock(mut self, clock: Arc<dyn Clock>) -> Self {
        self.clock = clock;
        self
    }

    pub fn with_limits(mut self, limits: Limits) -> Self {
        self.limits = limits;
        self
    }

    pub fn graph(&self) -> &Arc<Graph> {
        &self.graph
    }

    pub fn limits(&self) -> &Limits {
        &self.limits
    }

    /// Start a run. The returned stream drives execution when polled (no task is
    /// spawned), so it works on any executor. The final item is always a terminal
    /// event (`RunSucceeded`, `RunFailed`, `RunAborted` or `RunPaused`).
    pub fn run(&self, input: RunInput) -> RunStream {
        let queue = Arc::new(EventQueue::default());
        let control = Arc::new(Control::default());
        let sink_queue = Arc::clone(&queue);
        let sink: EventSink = Arc::new(move |event| sink_queue.push(event));
        let driver = root::run_root(self.clone(), input, control.clone(), sink).boxed();
        RunStream { driver: Some(driver), queue, finished: false, handle: RunHandle { control } }
    }

    /// Run to completion and return the terminal event, discarding the others.
    pub async fn run_to_end(&self, input: RunInput) -> Event {
        let mut stream = self.run(input);
        let mut last = None;
        while let Some(event) = stream.next().await {
            last = Some(event);
        }
        last.unwrap_or(Event::RunFailed {
            error: "run produced no events".to_owned(),
            error_type: "internal".to_owned(),
            usage: Default::default(),
            elapsed_ms: 0,
        })
    }
}

/// Events waiting for the stream's consumer.
///
/// Most events are produced while [`RunStream::poll_next`] polls the driver, on
/// the consumer's own task; nodes may also emit from tasks of their own (e.g. a
/// plugin's reader), which the waker covers. A `VecDeque` reuses its buffer, so
/// unlike a channel no allocation is made per event.
#[derive(Default)]
struct EventQueue {
    events: Mutex<VecDeque<Event>>,
    waker: AtomicWaker,
    /// Set when the stream is dropped: later events are discarded, not queued.
    closed: AtomicBool,
}

impl EventQueue {
    fn push(&self, event: Event) {
        if self.closed.load(Ordering::Acquire) {
            return;
        }
        self.events.lock().unwrap_or_else(PoisonError::into_inner).push_back(event);
        self.waker.wake();
    }

    fn pop(&self) -> Option<Event> {
        self.events.lock().unwrap_or_else(PoisonError::into_inner).pop_front()
    }
}

/// Stream of run events. `Send + 'static`. Dropping it cancels the run.
pub struct RunStream {
    driver: Option<BoxFuture<'static, ()>>,
    queue: Arc<EventQueue>,
    finished: bool,
    handle: RunHandle,
}

impl Drop for RunStream {
    fn drop(&mut self) {
        self.queue.closed.store(true, Ordering::Release);
    }
}

impl std::fmt::Debug for RunStream {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        f.debug_struct("RunStream").field("finished", &self.finished).finish_non_exhaustive()
    }
}

impl RunStream {
    /// A handle for controlling the run from elsewhere.
    pub fn handle(&self) -> RunHandle {
        self.handle.clone()
    }
}

impl Stream for RunStream {
    type Item = Event;

    fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Event>> {
        if self.finished {
            return Poll::Ready(None);
        }
        if let Some(driver) = self.driver.as_mut() {
            if driver.as_mut().poll(cx).is_ready() {
                self.driver = None;
            }
        }
        let event = match self.queue.pop() {
            Some(event) => Some(event),
            None if self.driver.is_none() => None,
            None => {
                // Only register when about to sleep (registering on every poll makes
                // events pushed by this very task wake it again), then look once more
                // so an event pushed from another task in between is not missed.
                self.queue.waker.register(cx.waker());
                match self.queue.pop() {
                    Some(event) => Some(event),
                    None => return Poll::Pending,
                }
            }
        };
        match event {
            Some(event) => {
                if event.is_terminal() {
                    self.finished = true;
                }
                Poll::Ready(Some(event))
            }
            None => {
                // Every event is sent synchronously, so a finished driver with an
                // empty queue means the run is over.
                self.finished = true;
                Poll::Ready(None)
            }
        }
    }
}

/// Controls a running run. Cheap to clone, `Send + Sync`.
#[derive(Clone, Debug)]
pub struct RunHandle {
    control: Arc<Control>,
}

impl RunHandle {
    /// Cancel immediately: in-flight nodes are dropped. Ends with `RunAborted`.
    pub fn abort(&self, reason: impl Into<String>) {
        self.control.stop(Stop::Abort(reason.into()));
    }

    /// Stop scheduling new nodes, let in-flight nodes finish, then end with `RunPaused`
    /// (or `RunSucceeded` if nothing was left to run).
    pub fn pause(&self, reason: impl Into<String>) {
        self.control.request_pause(Some(PauseReason {
            frame: Arc::from(""),
            node_id: None,
            kind: "external".to_owned(),
            data: Value::from(reason.into()),
        }));
    }

    /// Write a variable into the root frame (applied between node completions).
    pub fn update_variable(&self, selector: Selector, value: Value) {
        self.control.push_update(selector, value);
    }
}