Expand description
Pipes: connect one instance’s events to another instance’s input.
Today every server/client is an island. A pipe is a deterministic tap that
forwards one server’s network events into another server’s input, so an
operator (or the LLM, via a create_pipe action) can wire services together
without an LLM round-trip per hop.
The wiring is data, not code:
{ "from": <server_id>, "on": "http_request",
"to": <server_id>, "as": "send_tcp_data",
"map": { "data": "{method} {path}\n" } }When server from raises the on event, the map templates are rendered
against the event’s structured data ({method}, {path}, {headers.host},
…) and the resulting payload is delivered into server to. This reuses the
existing event system (crate::protocol::event_type::Event, tapped once in
call_llm) rather than adding any per-protocol glue.
§Delivery model (first slice)
A server has no inbound action executor — its input arrives on a real
socket — so a pipe feeds a server sink by opening a short TCP connection to
the target’s listen address and writing the mapped payload bytes. The payload
comes from the mapped data field, honouring an optional encoding
("utf8" default, or "hex"), exactly like send_tcp_data. This works for
any TCP-listening sink (TCP, and by raw bytes any text protocol); richer
per-protocol handshakes and client sinks (delivered through their own
outbound executor) are deliberately left as future work — see the module-level
notes in the task report.
§Safety
- Cycle refusal. Adding a pipe whose graph edge
from -> towould close a loop (including the self-loopA -> A) is refused bywould_create_cycle, so the obviousA -> B -> Aflood cannot be declared. - Bounded fan-out. Deliveries run under a process-wide semaphore
(
MAX_INFLIGHT_DELIVERIES). A high-rate source feeding a slow sink does not queue without bound: once the in-flight cap is reached, further deliveries are dropped with awarn!, never buffered, so memory stays bounded. Per-payload size is capped atMAX_PAYLOAD_BYTES. - Lifecycle. Pipes touching a server are torn down when that server
closes, mirroring scheduled-task scoping (see
AppStateInner::teardown_server).
Structs§
- Pipe
Spec - A declared pipe: forward
from’sonevents intotoasas_action, renderingmaptemplates against the event data.
Constants§
- MAX_
INFLIGHT_ DELIVERIES - Maximum concurrent in-flight pipe deliveries, process-wide. Deliveries beyond this are dropped-with-log rather than queued, so a slow sink cannot make a fast source exhaust memory.
- MAX_
PAYLOAD_ BYTES - Cap on a single rendered payload. A pipe is for log lines and small records, not bulk transfer.
- MAX_
RENDERED_ BYTES - Limit the intermediate template representation as well as the decoded payload. This allows separated hex while preventing repeated substitutions from allocating arbitrarily large strings before the payload limit can run.
Functions§
- dispatch_
pipes - Tap point: called once per event in
call_llm, before any handler runs, so a pipe fires regardless of how the source server answers its own peer (and even if the source’s LLM call later fails). - execute_
pipe_ action - Execute a
create_pipe/remove_pipeaction emitted by the LLM or an operator. Handled here (and dispatched from the action executor) rather than as aCommonActionvariant, keeping the enum and its many exhaustive matches untouched.default_fromis the server context the action ran in; acreate_pipemay omitfromto mean “this server”. - render_
payload - Render a spec’s mapping into the payload bytes to deliver to a socket sink.
- would_
create_ cycle - Would adding edge
from -> tocreate a directed cycle amongexistingpipes? Detects the self-loopfrom == toand anyto ->* frompath.
Type Aliases§
- PipeId
- Identifier for a pipe. Allocated from the same unified counter as servers and clients so ids never collide across kinds.