Crate kamaji_proto 

kamaji-proto — Yubaba ↔ Kamaji wire protocol.

Wire shape:

[u32 LE length][postcard-encoded payload]

The crate has no I/O. Callers (yubaba, kamaji) own the tokio::net::UnixStream and push/pull framed bytes through encode_frame / decode_frame.

Payload fields carrying a structure that accretes — the workload spec, the workload entries, the node capabilities — are encoded as a length-prefixed JSON blob inside that frame rather than inline (R896-F2). A peer one field behind skips a field it does not know instead of misreading every byte after it. See tolerant for the bound on which fields get this and why the frame and the greeting deliberately do not.

Both directions are versioned via ProtocolVersion; peers exchange YubabaToKamaji::Hello / KamajiToYubaba::Welcome at connection start. The handshake is exact equality, not negotiation: kamaji refuses any version that is not its own CURRENT (kamaji-bin/src/server.rs). This paragraph used to claim the receiver “picks the highest version it supports that the sender also offers” — it never did, and every operational note in the tree (scripts/hotship.sh, R876) describes the refusal instead. Corrected under R896-S1, because roll-safety reasoning was being done from the wrong sentence.

@arch:see(.yah/docs/working/W154-yubaba-dual-runtime.md)

@yah:relay(R896, “Evolvable kamaji wire envelope: stop the schema migrating into annotations”) @yah:at(2026-09-11T22:26:53Z) @yah:assignee(agent:user-custom-char-gul2) @yah:next(“From the 2026-09-11 yubaba/kamaji architecture review (chat session:d6fc1d54): the positional-postcard wire makes every WorkloadSpec field change a fleet-wide coordinated roll (R885-T6’s three-node bump), so new schema-shaped facts now land in annotations instead of fields — yah.limits., yah.durability., yah.placement.* — a stringly second WorkloadSpec that bypasses the JSON schema, TS export and serde validation, with R885-T6’s handoff naming the cost driver outright ("annotation not field, because a field costs exactly the wire bump"). Substrate MARKERS (yah.exec, yah.sandbox) are correct as annotations and stay. This track exists to make single-node hotships able to cross a field change, which is what keeps the chaos-survivability property cheap permanently.”) @arch:see(oss/yah-base/crates/workload-spec/src/lib.rs)

Re-exports

pub use codec::decode_frame;

pub use codec::encode_frame;

pub use codec::Error;

pub use codec::MAX_FRAME_BYTES;

pub use digest::spec_digest;

pub use digest::SpecDigest;

pub use messages::AckKind;

pub use messages::DrainBudget;

pub use messages::DrainOutcome;

pub use messages::DrainPhase;

pub use messages::ErrorCode;

pub use messages::ExitStatus;

pub use messages::KamajiToYubaba;

pub use messages::LogRecord;

pub use messages::LogStreamTag;

pub use messages::MeshAssignment;

pub use messages::MicroVmHealth;

pub use messages::NodeCapabilities;

pub use messages::ProbeStatus;

pub use messages::RequestId;

pub use messages::WireguardPeer;

pub use messages::WorkloadEntry;

pub use messages::WorkloadId;

pub use messages::WorkloadState;

pub use messages::YubabaToKamaji;

pub use version::ProtocolVersion;

Modules

codec

@yah:ticket(R590-B3, “kamaji UDS postcard decode fails on ImageRef untagged Deserialize — blocks all container deploys”) @yah:status(review) @yah:at(2026-07-03T07:05:47Z) @yah:assignee(agent:claude) @yah:parent(R590) @yah:severity(blocker) @yah:next(“ROOT CAUSE: workload_spec::ImageRef has a custom #[serde(untagged)] Deserialize (oss/yah-base/crates/workload-spec/src/lib.rs:1014, added R438-T3 for the string-form ‘reg/repo@sha256:…’ convenience). Untagged enums require serde deserialize_any, which postcard (non-self-describing) returns Error::WontImplement for. kamaji-proto/src/codec.rs:69 postcard::from_bytes the Workload::Container(spec) over the UDS → dies decoding the nested ImageRef.”) @yah:next(“FIX OPTION A (narrow, preferred): give the kamaji UDS a postcard-safe Workload DTO — encode ImageRef as its canonical docker_ref string (tag@digest) at the yubaba constable_client boundary + decode back, so the authoring-surface untagged convenience stays but never crosses postcard.”) @yah:next(“FIX OPTION B (wide): drop ImageRef’s untagged Deserialize; keep a plain derived struct Deserialize (postcard-safe) + a separate FromStr for TOML/recipe string-form. Bigger blast radius across recipe/compose authoring.”) @yah:next(“AFTER THE CODE FIX: the running yubaba+kamaji on us-west-002 is an OLD binary — the fleet must be REDEPLOYED with the fix before the live path goes green (ties into the warden→yubaba on-box redeploy gap).”) @yah:next(“Repro: cargo test -p yah –lib warden_client::tests::live_deploy_smoke – –ignored –nocapture (needs the mesh reachable).”) @yah:verify(“After fix + fleet redeploy: the live_deploy_smoke test deploys busybox to us-west-002 and reaches TERMINAL (not a postcard 500). Then a real forge/build-image workload runs on the node.”) @yah:gotcha(“Tier: Warrior — clear root cause + two concrete fix options, but touches the yubaba↔kamaji wire boundary (postcard DTO) with real blast-radius; needs careful implementation, not a rote edit.”) @yah:gotcha(“Surfaced by the R590-F2 live dogfood: deploying a real WorkloadSpec to us-west-002’s yubaba returns HTTP 500 ‘kamaji deploy_workload: kamaji error: Internal: decode failed: postcard error: This is a feature that PostCard will never implement’. R406-T9’s original ‘smoke’ only did curl GET /workloads (empty list) — a real deploy carrying an ImageRef through the postcard UDS was NEVER exercised, so this has been latent since kamaji’s Deploy arm landed.”) @yah:handoff(“DONE (Option 3, operator-chosen), verify-clean. The ticket’s root cause was INCOMPLETE: it was not just ImageRef’s untagged Deserialize. The whole workload_spec::Workload graph was postcard-decode-incompatible via TWO mechanisms, and a regression test (Workload::Container through postcard) surfaced both:\n(1) INTERNAL serde tagging on Workload + 11 nested enums (EnvValue/SecretRef/SecretTarget/VolumeSource/HealthProbe/RestartPolicy/PublicTls/BuildMode/AlmanacTarget/NotReadyPolicy/Cadence) -> #[serde(tag=…)] forces deserialize_any -> postcard WontImplement.\n(2) ~29 skip_serializing_if attrs -> postcard is positional, so an omitted None/empty field shifts every later field and decode dies with DeserializeBadOption. This one only bites when optionals are None; a full-spec test hid it, a for_forge (real forge deploy) spec exposes it.\nImageRef’s untagged Deserialize was a third instance.\n\nFIX (postcard-native end-to-end): flipped all 12 enums to external tagging; stripped all 29 skip_serializing_if (kept #[serde(default)] so hand-authoring can still omit fields; only machine-emitted JSON gains explicit null/[]); ImageRef Deserialize now branches on is_human_readable (string-form in TOML/JSON, plain struct in postcard). Regenerated TS (oss/packages/yah/workload-spec/index.ts) and JSON schema (.yah/schema/workload.toml.schema.json). Migrated every fixture: workload-spec crate (self) + oss/yubaba/cloud (16 reconciler tests, via a Sonnet subagent).\n\nVERIFY (all green): workload-spec full suite (incl. new round_trip postcard tests: full spec + all-None minimal spec + Workload::Container); kamaji-proto codec::deploy_container_round_trip (the exact wire path, was DeserializeBadOption); kamaji full workspace; yubaba –lib (cloud 486 pass); yah-base; schema_drift gate (2 pass). The live_deploy_smoke acceptance still needs the mesh + a FLEET REDEPLOY (old binary on us-west-002) – the wire decode is proven here; the on-box green is the redeploy step (ties to the warden->yubaba on-box redeploy gap).\n\nNOT-MINE pre-existing blockers found while verifying (each unrelated to this change): oss/qed test build fails on .unwrap() over impl Future (missing .await) in scryer/task_runs; oss/yubaba ServiceComponent missing field ‘git’ blocks whisper_derive_e2e + mesofact_static_e2e from compiling (whisper’s static-asset tagging was migrated but is unverifiable until that lands); root yah lib missing warden_client module file (warden->yubaba rename in flight). None block R590-B3.\n\nCascades: R592-T4 and R592-T5 (depends_on R590-B3) are now unblocked.”)

digest

Content digest of a deployed workload spec (R852-B4).

messages

@yah:ticket(R592-T4, “Finish warden/constable rename at the wire layer: enums, client type, socket defaults, unit templates”) @yah:status(review) @yah:assignee(agent:bundle-anthropic-ashguard) @yah:at(2026-07-06T07:43:16Z) @yah:phase(P3) @yah:parent(R592) @yah:verify(“grep for YubabaToKamaji / KamajiToYubaba / KamajiClient / run-constable paths in oss/kamaji returns zero hits; cd oss/kamaji && cargo test –workspace”) @yah:depends_on(R592-T1) @yah:depends_on(R590-B3) @yah:tier(Warrior) @yah:next(“DONE (R597-T1): env var renamed to KAMAJI_SOCK across kamaji-bin/src/main.rs, yah-yubaba/Dockerfile, yah-yubaba/pond-supervise.sh, kamaji.service comment.”) @yah:next(“DEFERRED -> filed as followup: yubaba-internal raft rename WardenState/WardenRequest/WardenNodeId/WardenRaft (oss/yubaba/crates/yubaba/src/raft/, leader.rs, lib.rs). Independent of the wire surface; postcard-internal.”) @yah:next(“OPTIONAL cosmetic: test file oss/yubaba/…/tests/integration_constable_client.rs keeps its old filename (content renamed to KamajiClient; git-mv skipped to avoid shared-tree churn).”) @yah:handoff(“DONE + verify-clean across 3 workspaces. Renamed the pub wire surface: WardenToConstable->YubabaToKamaji, ConstableToWarden->KamajiToYubaba, ConstableClient->KamajiClient, Welcome/ConstableInfo field constable_version->kamaji_version, plus stale doc module-path constable_proto::->kamaji_proto:: – across oss/kamaji (15 files), root crates/yah/hub (4), oss/yubaba (5). Postcard is positional so this is wire-compatible (no protocol-version bump). Socket PATHS already agreed everywhere (/run/kamaji/kamaji.sock – peer landed that under R589-T2 in commit e815d59), so no path edit needed; T4 shrank to the pure symbol rename.”) @yah:handoff(“INCIDENTAL green-keeping fix (NOT part of the rename): added render_command: None to two BuildConfig test fixtures (kamaji-proto/src/codec.rs:785, kamaji-bin/src/server.rs:761) that drifted when R535-T7 added BuildConfig.render_command (landed in the same e815d59 wip commit, fixtures not propagated). Two-line adaptation to unblock the workspace test.”) @yah:handoff(“VERIFY (all green): oss/kamaji cargo test --workspace 0 failed (kamaji-proto 24 + kamaji-bin lib 184 + sibling_wire_e2e 2 + uds_skeleton 1 + kamaji lib 29 + others); root cargo check -p hub --all-features clean; oss/yubaba cargo check -p yubaba --all-features + cargo test -p yubaba --test integration_constable_client --no-run compile clean. Grep: zero residual WardenToConstable/ConstableToWarden/ConstableClient/constable_version/constable_proto in all 3 workspaces; raft Warden symbols correctly untouched.”)

tolerant

Name-keyed payloads inside the postcard frame (R896-F2, W349).

version

@yah:ticket(R880-B1, “hotship’s proto-skew guard compares the tree against the last RELEASE, not against what the node runs — so it passes on a hotshipped node and breaks the kamaji/yubaba pair”) @yah:status(review) @yah:at(2026-10-07T16:30:28Z) @yah:assignee(agent:bundle-anthropic-ashguard) @yah:parent(R880) @yah:severity(high) @yah:next(“The guard reads proto_max() of the tree against proto_max() of the last release v* commit, on the stated assumption that "the node’s other half is on the released wire". That assumption is false on any node carrying a hotship — the normal state of this fleet. us-west-002/003 ran kamaji 0.8.40-h4 (unreleased) while v0.8.40 had just been cut, so tree-proto == release-proto, the guard saw no bump and said nothing, and the node’s h4 wire was older than both. Compare against the NODE: the script already probes each one and /health reports the other half’s version.\n<parameter name="assumes">Recovery used here was to ship the other half (–binaries kamaji) and then restart yubaba by hand on each node. yubaba’s KamajiClient connects ONCE at boot with a 30s budget and falls back permanently, so shipping kamaji alone does not re-pair. Whether hotship --binaries kamaji should also restart yubaba is undecided.\n<parameter name="verify">Point a tree whose kamaji_proto has moved past a node’s hotshipped half at that node with –binaries yubaba; today the guard is silent. After the fix it must refuse and name the node.”) @yah:gotcha(“Hit for real 2026-09-18. scripts/hotship.sh --nodes us-west-002,us-west-003 --binaries yubaba passed the guard and broke the kamaji UDS on both nodes: kamaji logged decode failed: postcard error: Serde Deserialization Error once per retry, and yubaba fell back to its in-process containerd runtime with a single WARN. The only external symptom is that GET /health then OMITS kamaji_version entirely — easy to read as transient. This is the exact failure the guard’s own R881-B7 comment predicts; the guard simply could not see it.”) @arch:see(scripts/hotship.sh) @yah:verify(“Point a tree whose kamaji_proto has moved past a node’s hotshipped half at that node with –binaries yubaba; today the guard is silent. After the fix it must refuse and name the node.”) @yah:handoff(“Guard now compares against each NODE. yubaba GET /health gains kamaji_protocol: u32 (ProtocolVersion::CURRENT.number(), new const fn in oss/kamaji/crates/kamaji-proto/src/version.rs). scripts/hotship.sh node_pair_proto probes every –nodes target (mesh then LAN): kamaji_protocol + kamaji_version present = pair proven on V; else UNPROVEN (old yubaba without the field, unreachable, or no kamaji handshaken — the 2026-09-18 symptom). Any mismatch or unproven node refuses a single-half restart ship and names the node; –no-restart warns; –allow-proto-skew overrides. Release-commit comparison removed. –allow-proto-skew help text updated.”) @yah:verify(“cd oss/kamaji && cargo test -p kamaji-proto –lib number_tests (1/1); cd oss/yubaba && cargo test -p yubaba –lib health (52/52); node_pair_proto exercised against stubbed /health bodies: proven→13, no kamaji_version→unproven, pre-field yubaba→unproven; bash -n clean. Not run against a live node.”) @yah:gotcha(“Rollout: no live node carries kamaji_protocol yet, so EVERY single-half hotship (–binaries yubaba or kamaji alone) refuses as unproven until a yubaba with this change is on that node — ship both halves once (–binaries kamaji,yubaba) per node, or –allow-proto-skew deliberately. Intended: unproven is never a match.”) @yah:handoff(“Operator call A (2026-10-07): a kamaji restart-ship without yubaba in –binaries now also restarts the node’s existing yubaba unit, so KamajiClient re-handshakes (scripts/hotship.sh unit:* activation arm). Uses the per-node raft floor already checked before the node; no sovereign-flag retire since yubaba bytes are unchanged. bash -n clean; not run against a live node.”)