Expand description
WorkloadSpec — typed wire format for yubaba workloads.
This crate is the schema source of truth. It has zero dependencies on yubaba; yubaba depends on it, not the other way around. Agents and desktop code that construct specs can link this crate without pulling in yubaba’s containerd client.
Three validation layers live in validate: shape (sync, no I/O),
semantic (reads yubaba state), and environment (deploy-time). The schema
types live at the top level.
@yah:ticket(R222-T3, “Workload schema doesn’t match per-kind on-disk shapes (mesofact-static)”) @yah:assignee(agent:claude) @yah:at(2026-05-18T16:47:03Z) @yah:status(review) @yah:parent(R222) @yah:handoff(“Picked option (a): tagged-enum Workload envelope with per-kind variants. Added Workload { MesofactStatic(MesofactStaticWorkload), Container(WorkloadSpec) } + BuildConfig in workload-spec. WorkloadSpec stays the containerd RPC wire type (now also the kind="container" variant payload). xtask emit-schemas now renders workload.toml.schema.json as a oneOf over kind; schema drift test green. TS export updated. Arch doc ‘workloads — colocated, not registered’ rewritten to describe the envelope + both example kinds; B4 outlook updated to point at Workload.”) @yah:verify(“cargo check -p cloud && cargo test -p cloud && cargo check -p yah && cargo check -p agent-tools && cargo check -p yah –tests && cargo check -p agent-tools –tests”) @yah:verify(“cargo test -p xtask # schema drift test must stay green”) @yah:verify(“cargo run -p workload-spec –bin export-ts # idempotent regen”)
@yah:ticket(R256-F7, “Model mesofact container as two roles: transient build/publish job vs long-lived SSR/SPA runtime”)
@yah:assignee(agent:claude)
@yah:at(2026-05-25T20:08:29Z)
@yah:status(review)
@yah:parent(R256)
@yah:next(“role A — build/publish job: transient task that runs the build and PUTs to the object store, then exits/GC’d; needed whenever there is a build step (SSR or not)”)
@yah:next(“role B — SSR/SPA runtime: long-lived container, only present when the app has realtime/dynamic pages (this is what the ‘only if SSR/SPA’ gate applies to)”)
@yah:next(“decide the fidelity knob: does the build run in-container (matches CI, max fidelity, costs image+cold-start) or on host with yubaba orchestrating only the serving edge?”)
@yah:assumes(“in cloud these are separate: CI/build job produces artifacts, R2+CDN serve them, and a distinct worker serves any SSR — so one merged ‘mesofact container’ is the trap”)
@yah:handoff(“BuildMode enum added to workload-spec with HostSide (default) and InContainer { image } variants. MesofactStaticWorkload gains build_mode: BuildMode (skip_serializing_if default) and ssr_runtime: Option
@yah:ticket(R256-F9, “Almanac as a dependency manifest: orchestrator verifies I/O targets live before run; output invalidates mesofact sources cleanly”)
@yah:assignee(agent:claude)
@yah:at(2026-05-25T21:28:15Z)
@yah:status(review)
@yah:parent(R256)
@yah:next(“almanac is a manifest declaring inputs + outputs + cadence + command — NOT a bash cron; the declared I/O is the contract”)
@yah:next(“before a run the orchestrator verifies declared inputs exist AND output targets (e.g. the mesofact app + its source/object store) are reachable; if not → the run fails or waits/times out rather than producing orphaned output”)
@yah:next(“almanac output invalidates downstream mesofact sources cleanly + deliberately (a declared dependency edge, not a blunt rebuild-everything) — this is the reason it’s a named manifest, not a shell cron”)
@yah:next(“decide the not-ready policy knob: fail-fast vs wait-with-timeout vs requeue”)
@yah:next(“generalizes the OpenRouter refresher (spawn_almanac_refresher), which is the degenerate no-dependency case (output = JSON cache, no app target)”)
@yah:assumes(“precondition enforcement lives in the shared scheduler layer (embedded by camp for dev/sim, yubaba for cloud/ha) — it needs the workload registry + xlb-net discovery to answer ‘is the target up?’, which a bash cron lacks”)
@arch:see(.yah/docs/architecture/A024-vocabulary.md)
@yah:depends_on(R256-F6)
@yah:handoff(“AlmanacTarget (Http/Tcp probe), NotReadyPolicy (WaitWithTimeout default=5s/FailFast/Requeue), Cadence (Once/Every/Cron), and AlmanacManifest types added to workload-spec. Workload enum gains Almanac(AlmanacManifest) variant (kind=‘almanac’). NotReadyPolicy::WaitWithTimeout(5s) is the default — matches sim-tier spinup budget. AlmanacManifest.invalidates: Vec
@yah:relay(R335, “Almanac mirror-binding — scope a feed to the mirror it affects”) @yah:at(2026-05-27T02:19:09Z) @yah:status(open) @arch:see(.yah/docs/working/W058-almanac-mirror-binding.md) @yah:depends_on(R256-F9)
@yah:ticket(R335-S1, “Decide cross-env pollution mechanism: extend R256-F9 manifest vs add per-mirror capability”) @yah:assignee(agent:claude) @yah:at(2026-05-27T02:19:33Z) @yah:kind(spike) @yah:status(review) @yah:phase(P1) @yah:parent(R335) @yah:gotcha(“Build ON R256-F9’s AlmanacManifest (workload-spec/src/lib.rs) — do NOT invent a parallel manifest. R256-F9 is in review.”) @yah:depends_on(R256-F9) @yah:handoff(“Decided. Recorded in almanac-mirror-binding.md §11. KEY FINDING: two almanac paths exist; the live R330 feed uses almanac::FeedConfig (on_change=MesofactRebuild{service,route} — a service id, NOT a MeshIdent), so it never touches AlmanacManifest.invalidates. Verdict on the S1 title: NEITHER extend the manifest nor (yet) add capability is the accident fix — dev->cloud is ALREADY blocked by construction (feed path = process locality + per-mirror reconciler + MinIO/R2 backend split; manifest path = no camp-embedded MeshState, mesh resolution is yubaba-raft-only). Residual holes: /revalidate receiver is UNAUTHENTICATED, and same-tier (two clouds on one R2) has no per-mirror key prefix.”) @yah:next(“FILED: R335-F3 (P1, no yubaba dep) mirror-aware /revalidate receiver — reject feeds not bound to this mirror; satisfies R335-T2; lands with R330-F4.”) @yah:next(“FILED: R335-F4 (P2) per-mirror artifact key prefix in derive_minio_key/publish_to_r2 — closes same-tier collision.”) @yah:next(“FILED: R335-F5 (P3, BLOCKED on yubaba control plane) per-mirror capability gate on /revalidate via yubaba/xlb-net node identity.”)
@yah:ticket(R278-F4, “RolloutPolicy schema in workload-spec (TOML types)”) @yah:assignee(agent:claude) @yah:at(2026-06-01T02:31:25Z) @yah:status(review) @yah:parent(R278) @yah:next(“Add src/rollout.rs with RolloutPolicy, RolloutStrategy, RolloutGate, RolloutStep, RolloutOnFailure”) @yah:next(“Export pub mod rollout from lib.rs”) @yah:next(“Add TS export via ts-rs in export-ts.rs”) @arch:see(.yah/docs/working/W140-yah-yubaba-ci-cd.md) @yah:handoff(“RolloutPolicy, RolloutStrategy, RolloutGate, RolloutStep, RolloutOnFailure added to workload-spec/src/rollout.rs. Exported from lib.rs. toml dev-dep added for round-trip test. Tests: rollout::tests::round_trip_toml + on_failure_default both green.”)
@yah:ticket(R429-T1, “Workload::StaticAsset variant + schema in workload-spec (catalog + aliases)”)
@yah:assignee(agent:claude)
@yah:at(2026-06-03T23:24:20Z)
@yah:status(review)
@yah:phase(P1)
@yah:parent(R429)
@yah:next(“Add Workload::StaticAsset(StaticAssetWorkload) variant alongside the existing MesofactStatic + Container envelopes. Mirror the tagged-enum shape R222-T3 established.”)
@yah:next(“StaticAssetWorkload fields: kind=‘static-asset’ tag, assets: Vec
@yah:ticket(R429-F2, “static-asset reconciler: BLAKE3 verify + S3 PUT against mirror’s object_store + drift”)
@yah:assignee(agent:claude)
@yah:at(2026-06-03T23:24:38Z)
@yah:status(review)
@yah:phase(P2)
@yah:parent(R429)
@yah:next(“New reconciler that handles kind=‘static-asset’ in the same service-sync loop that already runs mesofact-static + container. Same wave-gate semantics, same drift shape.”)
@yah:next(“For each [[asset]] row: hash source file (BLAKE3) and compare to manifest entry. Mismatch → surface as drift, halt push for that asset until rebuild.”)
@yah:next(“Resolve mirror’s object_store provider → R2 bucket + credentials. HEAD cas/filename; if absent or different content-length → PUT. Idempotent on re-run.”)
@yah:next(“Drift detection: list bucket contents under the component’s prefix, compare against catalog filenames. Files in bucket ∖ catalog → report as drift (do NOT delete; that’s the prune verb’s job).”)
@yah:next(“ServicesView’s existing matrix consumes the new drift shape automatically. Confirm SyncGlyph/DriftList render correctly for a static-asset row without UI changes.”)
@yah:next(“MockR2 in tests: HashMap<key, bytes> implementing the S3 surface the reconciler hits. Cover: push first-time, push idempotent, drift catches catalog-vs-bucket mismatch, BLAKE3 mismatch halts push.”)
@yah:next(“Real-R2 integration test gated behind YAH_TEST_R2_BUCKET env var — one round-trip against a scratch bucket; skipped otherwise.”)
@yah:verify(“cargo check –workspace –locked”)
@yah:verify(“cargo test -p
@yah:ticket(R429-T3, “yah service prune verb: candidate enumeration + operator-confirm delete”)
@yah:assignee(agent:claude)
@yah:at(2026-06-03T23:24:52Z)
@yah:status(review)
@yah:phase(P3)
@yah:parent(R429)
@yah:next(“yah service prune yah cloud service prune <name> --env <env> [--dry-run] [--yes] [--format=table|json] at app/yah/cli/src/cloud.rs (ServiceCommands::Prune + handle_service_prune). MCP tools cloud.service_prune_status (read, auto-pass, –dry-run –format=json) and cloud.service_prune (write, –yes –format=json) dispatch through build_command(). New S3 helper sign_s3_get_with_query in local-driver covers ListObjectsV2 (the existing s3_sign helpers don’t handle canonical query strings); ListObjectsV2 response is parsed with a tiny hand-rolled split_tags helper to avoid a quick-xml workspace dep. Tests: 12 prune-module unit tests (live-set union, kind filtering, list response parse for single/empty/truncated/no-token, candidate filtering including catalog manifest sidecar exclusion) + 1 s3_sign helper test + 2 MCP build_command tests. cargo check –workspace clean. cargo test -p cloud –lib: 279 pass (1 pre-existing failure cloud_init::tests::embedded_template_matches_workspace_canonical unrelated, per R419-F4 docstring). cargo test -p yah –lib: 299 pass.”)
@yah:next(“R429-F4 carries the Tauri + DeployPanel UI work — depends_on R429-T3, status=open.”)
@yah:verify(“cargo check –workspace –locked”)
@yah:verify(“cargo test -p cloud –lib reconciler::static_asset_prune # 12 pass”)
@yah:verify(“cargo test -p yah –lib mcp::tools::tests::cloud_service_prune # 2 pass”)
@yah:verify(“yah cloud service prune –help # renders usage with –env/–dry-run/–yes/–format”)
@arch:see(.yah/docs/working/W164-derived-static-assets.md)
@yah:ticket(R438-T2, “AssetEntry XOR: source vs derive + shape_static_asset rules”)
@yah:assignee(agent:claude)
@yah:at(2026-06-04T21:06:51Z)
@yah:status(review)
@yah:phase(P1)
@yah:parent(R438)
@yah:next(“AssetEntry.source: PathBuf → Option
@yah:ticket(R438-T3, “ImageRef digest-pin enforcement at deserialize”)
@yah:assignee(agent:claude)
@yah:at(2026-06-04T21:06:55Z)
@yah:status(review)
@yah:phase(P1)
@yah:parent(R438)
@arch:see(.yah/docs/working/W164-derived-static-assets.md)
@arch:see(.yah/docs/working/W165-mesofact-build-mode-lowering.md)
@yah:handoff(“ImageRef now accepts either a string form (digest-pinned, W164/W165 path) or the legacy struct form (backwards-compat for WorkloadSpec configs). String form requires @sha256:
@yah:ticket(R438-T7, “Golden tests: recipe→ForgeSpec lowering + BuildMode→ForgeSpec lowering parity”)
@yah:assignee(agent:claude)
@yah:at(2026-06-04T21:07:30Z)
@yah:status(review)
@yah:phase(P3)
@yah:parent(R438)
@yah:next(“Golden test: sample transform recipe + asset.derive.transform.params lowers to expected ForgeSpec (argv, image digest, TaskPlacement)”)
@yah:next(“Golden test: MesofactStaticWorkload with build_mode=in_container lowers to expected ForgeSpec”)
@yah:next(“Round-trip parity: same Subprocess + Local + Container quadrant for both consumers; regression-guards argv-substitution and image-pin drop-through”)
@yah:verify(“cargo test -p workload-spec lowering_golden_*”)
@yah:verify(“Golden files versioned; updates require explicit –update flag”)
@arch:see(.yah/docs/working/W164-derived-static-assets.md)
@arch:see(.yah/docs/working/W165-mesofact-build-mode-lowering.md)
@yah:depends_on(R438-T5)
@yah:depends_on(R438-T6)
@yah:handoff(“T7 landed. (1) Extracted pure lowering helpers exposed at pub(crate):\n - mesofact_static::lower_build_to_forge_spec(workload_dir, &BuildConfig, &BuildMode) -> ForgeSpec (run_build now wraps this)\n - static_asset::lower_recipe_step_to_forge_spec(&TransformRecipe, &RecipeStep, substituted_argv) -> ForgeSpec (materialize_transform now calls this for each step)\n(2) New cfg(test) module crates/yah/cloud/src/reconciler/lowering_golden.rs registered from reconciler/mod.rs. Five golden tests:\n - golden_recipe_step_lowers_to_pinned_local_container_subprocess (recipe → ForgeSpec shape: argv, image digest, timeout, label, initiator)\n - golden_recipe_step_with_zero_timeout_lowers_to_none (regression-guards the timeout=0 → None mapping)\n - golden_build_in_container_lowers_to_pinned_local_container_subprocess (BuildMode::InContainer → sh -c shell wrap + pinned image + cwd label)\n - golden_build_host_side_lowers_to_native_quadrant_without_image (BuildMode::HostSide → image=None + TaskRuntime::Native)\n - parity_recipe_and_build_in_container_share_quadrant (THE architectural invariant: both consumers land in the same Subprocess + Local + Container quadrant with sha256-pinned images and Gnome initiators — lets one ForgeExecutor dispatch handle both)\n(3) Test artifacts are hand-coded assertions, not insta/snapshot files — workspace has no insta infra and explicit-Pin tests give clearer diff on drift than auto-update snapshots. The W164/W165 lowering shape is now regression-guarded against silent drift in either consumer. cargo test -p cloud –lib reconciler::lowering_golden: 5 pass. Workspace check clean.”)
@yah:next(“Sign off → archive R438-T7”)
@yah:next(“T8 (worked examples) now has tested lowering primitives to reference”)
@yah:verify(“cargo test -p cloud –lib reconciler::lowering_golden — 5 pass”)
@yah:verify(“cargo test -p cloud –lib reconciler:: — 124 pass; 4 pre-existing R441-B4 adopt_only failures (port 4321 dev-box collision) unrelated”)
@yah:verify(“cargo check –workspace –locked — clean (warnings only)”)
@yah:verify(“Parity test asserts both lowerings produce TaskPlacement{Local, Container} + ForgeCommand::Subprocess + sha256-pinned image — the shared executor dispatch invariant”)
@yah:gotcha(“Test location pivot: original ticket said cargo test -p workload-spec lowering_golden_* but the lowering primitives don’t live in workload-spec — ForgeSpec/TaskPlacement are in task, and the actual lowering helpers are in cloud (both consumers live there). Tests landed in cloud as reconciler::lowering_golden. If a future consumer outside cloud needs the BuildMode lowering, lift lower_build_to_forge_spec up to task::transforms alongside the existing recipe lowering primitives.”)
@yah:gotcha(“No snapshot/insta infra in workspace — ‘Golden files versioned; updates require explicit –update flag’ verify line interpreted as hand-coded explicit assertions instead. Drift surfaces as a single-file test diff on the lowering helper, which is more readable than a .snap diff for the small ForgeSpec shape these tests cover.”)
Modules§
- compose_
import - One-way, lossy compose YAML →
WorkloadSpecimport shim. - rollout
- Rollout policy schema — the typed form of
.yubaba/rollout.toml. - secrets
- Pluggable secret resolver for
crate::SecretRefvalues. - validate
- Shape validators for
WorkloadSpec.
Structs§
- Almanac
Manifest kind = "almanac"manifest — a declared data-pipeline job.- Asset
Derive - Provenance chain for a derived asset: required
fetchstep, optionaltransformstep. Materialized bytes replaceAssetEntry.sourcefor the rest of the static-asset reconcile loop. - Asset
Entry - A single file entry in the static-asset catalog.
- Backoff
Policy - Exponential backoff parameters for
RestartPolicy::OnFailure. - Blake
Hash - BLAKE3 content hash expressed as exactly 64 ASCII hex digits.
- Build
Config - Build step that produces the static artifact published by a
mesofact-staticworkload. - Derive
Lock - W212/R518: the committed derivation lock — the in-tree action-cache
receipt.
input_hashis the input-addressed derivation key computed over the complete declared input set (fetched-input pin ⊕ recipe-file bytes ⊕ invocation params ⊕ schema version);output_blake3is what those inputs produced (== the entry’sblake3). The reconciler skips the entire build (no fetch, no transform, no PUT) when the lock matches the inputs recomputed from the current pins and the bucket already holds the output — the Nix-substituter / Bazel-remote-cache behaviour. Written by the R510 bind path from the reconciler’sdiscovered_input_hash:<filename>output; thegit diffon this block is the receipt that the derivation rolled. - EnvVar
- A single environment variable injected into the container.
- Expose
Spec - Network exposure configuration. The three channels are independent; any combination is valid.
- Fetch
Source - Shared fetch primitive — usable by
asset.derivetoday, and by Almanac’sReleaseSourceafter a follow-up migration (R438-F10). Defined once in workload-spec so both consumers reject the same set of non-permissive licenses. - Healthcheck
- Container health probe configuration.
- Image
Ref - Container image reference identifying a specific image to pull.
- Machine
Id - Opaque identifier for a yubaba-managed machine within the cluster.
- Mesh
Expose - Mesh-internal port exposure and peer access control.
- Mesh
Ident - DNS-segment identity for a workload on the cluster mesh, e.g.
"noisetable-api.pdx". Regex constraint:^[a-z0-9]([a-z0-9-]*[a-z0-9])?$, length ≤ 63. Enforced in shape validation (R090-F2). - Mesofact
Static Workload kind = "mesofact-static"payload — static-site build colocated with the frontend it deploys.- Millis
- Duration expressed as an integer millisecond count.
- Operator
Expose - Operator-facing exposure via a Tailscale ACL tag.
- Public
Expose - Public internet exposure via a Cloudflare tunnel route.
- Resource
Limits - Hard resource caps enforced by containerd/cgroups at runtime.
- Secret
Mount - A secret value mounted into the container as an env var or file.
- Static
Asset Workload kind = "static-asset"payload — content-addressed bucket catalog.- Stop
Policy - Graceful shutdown configuration for yubaba’s stop sequence.
- TierTag
- Tier classification that governs admission control and mesh
allow_fromfiltering. Known values:"public","tenant","private","infra". Custom tiers are allowed per cluster; shape validation warns on unknowns rather than rejecting them (R090-F2). - Transform
Spec - Optional transform applied after a
FetchSourcedownload, lowering to aForgeCommand::Subprocessvia the recipe loader (R438-T4). The transform’s output is content-addressed by the entry’sblake3(the recipe runs only when the cache misses). - Volume
Mount - A volume mount inside the container.
- Workload
Spec - Complete typed description of a containerd workload handed to yubaba over
RPC. This is also the payload of the
kind = "container"variant ofWorkloadon disk.
Enums§
- Almanac
Target - An observable endpoint the almanac scheduler probes to check readiness.
- Build
Mode - Where the build command runs for a
mesofact-staticworkload. - Cadence
- When the almanac scheduler triggers a run.
- EnvValue
- Value source for an environment variable.
- Health
Probe - Mechanism used to check container health.
- License
- Closed-set, parse-time-enforced license tag. Mirrors the workspace
permissive-license rule (MIT / Apache-2.0 / BSD-2/3-Clause / ISC). Adding a
variant is an explicit schema change — non-permissive strings
(
"GPL-3.0","AGPL", etc.) fail at serde-deserialize before any shape validator runs. - Mesh
Lookup - Which aspect of a mesh peer’s address to inject.
- NotReady
Policy - What the almanac scheduler does when a precondition check fails.
- Public
Tls - TLS mode for a public endpoint.
- Restart
Policy - What yubaba does when the container exits.
- Schema
Version - Wire-format schema version envelope.
- Secret
Ref - Where yubaba resolves the secret value from.
- Secret
Target - How the secret is surfaced inside the container.
- Volume
Source - Backing source for a volume mount.
- Workload
- On-disk
workload.tomlmanifest. Each variant matches oneServiceComponent.kindvalue; thekindfield on the wire is the serde discriminator.
Constants§
- HOST_
NETWORK_ ANNOTATION - Annotation key requesting a workload share the host network namespace.
See
WorkloadSpec::wants_host_network. - HOST_
NETWORK_ VALUE - Annotation value (for
HOST_NETWORK_ANNOTATION) selecting host networking. Any other value leaves the workload in an isolated netns.
Traits§
- Workload
Runtime - Shared interface for deploying and managing
WorkloadSpeccontainers.