Skip to main content

Module pipe

Module pipe 

Source
Expand description

Pipes: connect one instance’s events to another instance’s input.

Today every server/client is an island. A pipe is a deterministic tap that forwards one server’s network events into another server’s input, so an operator (or the LLM, via a create_pipe action) can wire services together without an LLM round-trip per hop.

The wiring is data, not code:

{ "from": <server_id>, "on": "http_request",
  "to": <server_id>, "as": "send_tcp_data",
  "map": { "data": "{method} {path}\n" } }

When server from raises the on event, the map templates are rendered against the event’s structured data ({method}, {path}, {headers.host}, …) and the resulting payload is delivered into server to. This reuses the existing event system (crate::protocol::event_type::Event, tapped once in call_llm) rather than adding any per-protocol glue.

§Delivery model (first slice)

A server has no inbound action executor — its input arrives on a real socket — so a pipe feeds a server sink by opening a short TCP connection to the target’s listen address and writing the mapped payload bytes. The payload comes from the mapped data field, honouring an optional encoding ("utf8" default, or "hex"), exactly like send_tcp_data. This works for any TCP-listening sink (TCP, and by raw bytes any text protocol); richer per-protocol handshakes and client sinks (delivered through their own outbound executor) are deliberately left as future work — see the module-level notes in the task report.

§Safety

  • Cycle refusal. Adding a pipe whose graph edge from -> to would close a loop (including the self-loop A -> A) is refused by would_create_cycle, so the obvious A -> B -> A flood cannot be declared.
  • Bounded fan-out. Deliveries run under a process-wide semaphore (MAX_INFLIGHT_DELIVERIES). A high-rate source feeding a slow sink does not queue without bound: once the in-flight cap is reached, further deliveries are dropped with a warn!, never buffered, so memory stays bounded. Per-payload size is capped at MAX_PAYLOAD_BYTES.
  • Lifecycle. Pipes touching a server are torn down when that server closes, mirroring scheduled-task scoping (see AppStateInner::teardown_server).

Structs§

PipeSpec
A declared pipe: forward from’s on events into to as as_action, rendering map templates against the event data.

Constants§

MAX_INFLIGHT_DELIVERIES
Maximum concurrent in-flight pipe deliveries, process-wide. Deliveries beyond this are dropped-with-log rather than queued, so a slow sink cannot make a fast source exhaust memory.
MAX_PAYLOAD_BYTES
Cap on a single rendered payload. A pipe is for log lines and small records, not bulk transfer.
MAX_RENDERED_BYTES
Limit the intermediate template representation as well as the decoded payload. This allows separated hex while preventing repeated substitutions from allocating arbitrarily large strings before the payload limit can run.

Functions§

dispatch_pipes
Tap point: called once per event in call_llm, before any handler runs, so a pipe fires regardless of how the source server answers its own peer (and even if the source’s LLM call later fails).
execute_pipe_action
Execute a create_pipe / remove_pipe action emitted by the LLM or an operator. Handled here (and dispatched from the action executor) rather than as a CommonAction variant, keeping the enum and its many exhaustive matches untouched. default_from is the server context the action ran in; a create_pipe may omit from to mean “this server”.
render_payload
Render a spec’s mapping into the payload bytes to deliver to a socket sink.
would_create_cycle
Would adding edge from -> to create a directed cycle among existing pipes? Detects the self-loop from == to and any to ->* from path.

Type Aliases§

PipeId
Identifier for a pipe. Allocated from the same unified counter as servers and clients so ids never collide across kinds.