datum-agent

datum-agent is the local operations satellite for Datum. It provides an embeddable job registry actor for named, supervised, materialized streams, including graceful drain, hard stop, restart with backoff, lifecycle status snapshots, and a stream-native event feed.

It also provides the Datum Control Protocol (DCP) server/client layer used by the daemon, CLI, and cluster control plane. DCP is prost over length-prefixed TCP/QUIC frames: QUIC with mTLS is the remote deployment path, and plaintext TCP is accepted only for loopback local development. The DCP major version is negotiated during Hello; unknown major versions are rejected. Minor version bumps are additive within the same major, so new request variants such as ListClusterJobs and ClusterNodeInfo do not change the major.

StartJob uses a registered-factory model: StartJob { factory_name, instance_name, params }. The daemon registers named JobSpec factories at startup; closures and stream blueprints do not cross the wire. Cluster placement uses the same registered factories; dynamic blueprint upload remains out of scope.

Cluster-aware agents

ClusterAgent starts the local Agent, DCP server, datum-cluster membership node, and DCP node-session manager together. Members that are Up and advertise the agent role maintain single-owner async DCP sessions to each other. QUIC+mTLS is the remote transport; loopback TCP is available for tests and local development.

Cluster DCP requests now include WP-C3 placement. SubmitClusterJob { factory_name, instance_name, params, placement } can be sent to any cluster-aware agent; non-coordinators forward it over their C2 node session to the deterministic placement coordinator. The coordinator is the oldest Up member in the local C1 view, tie-broken by node id, with the same split-brain caveat as timeout downing: during a partition, both sides can transiently place jobs until v0.11+ adds quorum or lease fencing.

PlacementSpec v1 supports an optional role constraint and either LeastJobs or Pinned(node_id). The coordinator starts the selected registered factory on the target node, stores the assignment in memory, and replicates the assignment best-effort to peers so the next coordinator can take over. On takeover it rebuilds the placement table by querying local registries across surviving nodes. There is no persistent placement store in v0.10.

When a member with active cluster-submitted jobs reaches Down, the coordinator starts fresh placement generations on surviving eligible nodes. This is at-least-once materialization: the downed node may still be running its old copy until it observes its own downing. Fencing is a named follow-up. Jobs started directly on one agent without cluster placement metadata are local-only and are never re-placed.

ListClusterJobs returns local and peer registry snapshots with placement metadata, generation, and history for cluster jobs. DrainJob, StopJob, and JobStatus can be marked cluster-scoped; they route through the coordinator to the current placement node. ClusterNodeInfo returns the local membership view plus each peer session state.

The datum-agent binary enables this wiring with --cluster or DATUM_AGENT_CLUSTER=1. Its simple config file accepts a [cluster] section for keys such as node_id, roles, bind_addr, advertise_addr, seeds, dcp_transport, and fan-out/reconnect timeouts; [dcp] maps to the existing DCP listener keys.