# Public API
```rust
use std::{future::Future, pin::Pin, sync::{Arc, atomic::{AtomicU8, AtomicU64}}};
use rust_decimal::Decimal;
pub type LlmFuture<'a> = Pin<Box<dyn Future<Output = Result<Inference, LlmError>> + Send + 'a>>;
pub type ToolFuture = Pin<Box<dyn Future<Output = ToolOutput> + Send + 'static>>;
pub type CompactFuture = Pin<Box<dyn Future<Output = Result<String, String>> + Send + 'static>>;
pub trait Llm: Send + Sync {
fn start(&self) -> Box<dyn LlmThread>;
}
pub trait LlmThread: Send {
fn infer<'a>(&'a mut self, delta: &'a str) -> LlmFuture<'a>;
}
#[derive(Clone, Debug, Eq, PartialEq)]
pub enum LlmError { Transient(String), Permanent(String) }
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct Inference {
pub text: String,
pub calls: Vec<Call>,
pub continue_inference: bool,
}
#[derive(Clone, Debug, Eq, PartialEq)]
pub enum Call { Tool(ToolRequest), Worker(WorkerRequest), Compact(CompactRequest) }
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct ToolRequest { pub name: String, pub input: String }
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct WorkerRequest { pub llm: String, pub prompt: String }
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct CompactRequest { pub instruction: String }
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct ToolOutput { pub text: String, pub cost_cents: Decimal }
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum ToolMode { Fast, Queued }
pub struct ToolStart { pub mode: ToolMode, pub queued: String, pub future: ToolFuture }
pub struct WorkerStart { pub llm: Arc<dyn Llm>, pub queued: String }
pub trait Runtime: Send + Sync {
fn start_tool(&self, request: ToolRequest, updates: Updates) -> Result<ToolStart, String>;
fn start_worker(&self, request: &WorkerRequest) -> Result<WorkerStart, String>;
fn compact(&self, request: CompactRequest, primary: String) -> CompactFuture;
}
#[derive(Clone, Copy, Debug, Eq, Hash, Ord, PartialEq, PartialOrd)]
pub struct ActionId { /* private fields */ }
impl ActionId {
pub const fn new(session: [u8; 12], sequence: u64) -> Self;
pub const fn session(self) -> [u8; 12];
pub const fn sequence(self) -> u64;
}
#[derive(Clone, Debug, Eq, PartialEq)]
pub enum SubmittedUpdate { Activity(String), Append(String) }
pub trait UpdateSink: Send + Sync + 'static {
fn submit(&self, action: ActionId, identity: u64, update: SubmittedUpdate) -> Result<(), ChatError>;
}
#[derive(Clone)]
pub struct Updates { /* private fields */ }
impl Updates {
pub fn bind(action: ActionId, sink: Arc<dyn UpdateSink>) -> Self;
pub const fn action_id(&self) -> ActionId;
pub fn activity(&self, text: String) -> PreparedUpdate;
pub fn append(&self, text: String) -> PreparedUpdate;
}
#[derive(Clone)]
pub struct PreparedUpdate { /* private fields */ }
impl PreparedUpdate {
pub fn send(&self) -> Result<(), ChatError>;
}
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct ChatView {
pub primary: String,
pub pending: String,
pub history: Vec<String>,
pub actions: Vec<PendingAction>,
}
#[derive(Clone, Debug, Eq, PartialEq)]
pub enum PendingAction {
Inference { attempt: u8 },
Tool { name: String },
Worker { llm: String },
Compaction,
}
#[derive(Clone, Debug, Eq, PartialEq)]
pub enum ChatEvent { Text(String), Activity(String), Stalled(String) }
#[derive(Clone, Debug, Eq, PartialEq)]
pub enum ChatError { Empty, NotStalled, Busy, Closed }
pub async fn infer_with_retry(
thread: Box<dyn LlmThread>,
delta: String,
attempt: Arc<AtomicU8>,
) -> (Box<dyn LlmThread>, Result<Inference, String>);
```
`ToolOutput::text` is terminal tool text and `cost_cents` is its exact decimal cost in cents; free tools use zero. `Runtime` setup callbacks are nonblocking, with remote or expensive work in returned futures.
`ActionId` contains exactly a 12-byte session identity and one action sequence; its methods perform no parsing, formatting, allocation, randomness, or validation. Each `Updates::bind` starts update identity allocation at one. Its clones share one relaxed `AtomicU64`, so distinct preparations receive distinct identities. Cloning a `PreparedUpdate` retains its action ID, update identity, payload, and sink. Every send invokes the sink; downstream state deduplicates by action and update identity. The wrapper creates no task, channel, wait, retry, or timeout.
`infer_with_retry` invokes the same thread with the identical delta up to five times. Immediately before each call it stores attempt 1 through 5 with relaxed ordering. Success or permanent failure returns immediately. Transient failures after attempts one through four wait 10, 20, 40, and 80 seconds; the fifth transient returns without another wait. An inference call has no timeout and may remain pending indefinitely. Calls never overlap, retries never replace the thread, and dropping the operation drops the active inference or timer without returning the thread.
- `ActionId::new`: Performance: Not yet benchmarked; work and allocation are constant.
- `ActionId::session`: Performance: Not yet benchmarked; work and allocation are constant.
- `ActionId::sequence`: Performance: Not yet benchmarked; work and allocation are constant.
- `Updates::bind`: Performance: Not yet benchmarked; work and retained memory are constant apart from the supplied sink.
- `Updates::action_id`: Performance: Not yet benchmarked; work and allocation are constant.
- `Updates::activity`: Performance: Not yet benchmarked; work and allocation are constant apart from retaining the supplied text.
- `Updates::append`: Performance: Not yet benchmarked; work and allocation are constant apart from retaining the supplied text.
- `PreparedUpdate::send`: Performance: Not yet benchmarked; Kennedy-owned work and temporary memory are linear in payload bytes before one synchronous sink call, whose completion is sink-controlled.
- `infer_with_retry`: Performance: Not yet benchmarked; Kennedy-owned work is constant per call and timer while up to five sequential inference attempts own completion latency and output memory.