consortium-builder

Compiles, post-processes, and stages every artifact a Consortium.toml describes.

consortium-cfg answers "what does this system look like, and is it valid?" and emits generated source. This crate answers "now actually build it": four compilers, a post-processing step remoteproc requires, and a staging tree a packaging recipe can consume.

It stops short of deployment. Nothing here writes to a live target filesystem — the output is a local dist/ tree plus an optional BitBake hand-off.

The user-facing entry point is csti build; this crate is the library behind it.

Pipeline

Consortium.toml
  ├─ 1. firmware      cargo, bare-metal target → ELF → rename .vector_table
  ├─ 2. elf_map       the defmt map the app embeds for decoding
  ├─ 3. hmi           vp / npm / bun → frontend bundle
  ├─ 4. tee           OP-TEE TA dev kit → signed <uuid>.ta
  ├─ 5. application   cargo, usually cross-compiled → Linux executable
  ├─ 6. systemd       render the unit
  ├─ 7. dist          assemble dist/ + a BitBake recipe
  └─ 8. bitbake       optional stage / invoke / dry-run hand-off

Plan, then run

pipeline::plan resolves every stage's inputs — paths, env vars, artifact locations — without running any build, threading the data flow between stages. pipeline::run plans, then executes.

That split is what makes the pipeline debuggable. A wrong sysroot or a missing firmware ELF surfaces as a planning failure naming the stage, rather than as a cargo error several minutes into a cross-compile.

The Compiler trait

Firmware, the Linux application, the OP-TEE TA, and the HMI frontend are built by very different toolchains:

Compiler makes them uniform: each implementor owns a Configuration carrying all its parameters, a typed Error, and an Output. Its methods take the config rather than &self, so every implementor has the same .build(cfg) / .output_path(&cfg) shape and the pipeline needs no per-kind special casing.

Compile-time configuration

Memory layout lives in generated source, not the environment. Controller firmware gets its IPC region base and its [dbg] ring baked into generated init(), and the Linux application reaches the same memory through its UIO devices, so there are no CONSORTIUM_IPC_* or CONSORTIUM_DBG_* variables.

What still arrives as environment variables is what generated code cannot know, read with option_env!/env!: the on-target HMI asset paths (compile::env::HmiConfig), the TEE UUID, and the CSTI_* build-host directories (CstiWorkspace). Numeric values are emitted as decimal strings.

Things that are easy to get wrong

Firmware post-processing is mandatory. Linux remoteproc expects a coprocessor's vector table in a section named .isr_vectors, but cortex-m-rt emits .vector_table. The pipeline renames it and keeps the original alongside the deployable ELF — an ELF that boots and an ELF a debugger understands are not quite the same file.

Sysroot resolution has an order. [profile] sysroot first, then SDKTARGETSYSROOT / OECORE_TARGET_SYSROOT, so a manifest can pin a sysroot without the developer having sourced an SDK script. On an arm64 Linux host, aarch64-unknown-linux-gnu is a native build and sysroot = "/" resolves to the image root — which is how the melt-pot manifests build inside an Apple container VM with no SDK at all.

TA signing is optional on purpose. Signing needs the dev kit's signer script and keys, which a laptop or CI runner usually lacks, so the unsigned ELF can be staged instead (--no-sign-ta). Compiling the TA and signing it are separable steps.

Linux device-tree sources are external inputs. csti build --linux <DIR> selects the BSP .dts/.dtsi, follows its local includes, and supplies dt-bindings/*.h for dtc. Consortium bundles none of those Linux sources.

The HMI build command can be overridden. CONSORTIUM_HMI_BUILD_CMD takes precedence over vp/npm. That exists because the frontend sometimes has to be built elsewhere: inside a Linux container the host's node_modules/ is already populated, so pointing the command at /bin/true lets the pipeline consume the existing dist.

Staging

The assembled tree can contain app/, firmware/, ta/, www/, dts/linux/, kernel/, boot/, and systemd/, plus a generated BitBake recipe.

Firmware ELFs land at dist/firmware/<core>.elf so a firmware-packaging recipe can install them to /lib/firmware/<core> for remoteproc's kernel-side loader. That is unrelated to the defmt ELF map, which is embedded in the application binary.

[profile] deploy_root is a naming convention only — it composes the HMI dist path, the systemd unit's ExecStart=/WorkingDirectory=, and the firmware install location a recipe consumes. The pipeline never writes there.

BitBake hand-off

Staging (--bitbake-files-dir) and invoking (--bitbake-recipe + --bitbake-build-dir) compose: given both, dist/ is staged into the recipe's files/ directory first, then BitBake runs. --bitbake-dry-run records the command without running it — that is the split-machine flow, where dist/ is produced on one host and the Yocto build happens on another.

Extending

Prefer extending consortium-cfg lowering first, then this pipeline. When working here, keep the four boundaries distinct: planning, compilation, staging, and the external BitBake hand-off. Mixing them is what makes a build pipeline untestable.

progress::BuildReporter is deliberately minimal — library callers can ignore it and keep inherited subprocess output, while the CLI passes a reporter that holds a persistent step prompt under cargo/vp/bitbake output.

Testing

cargo test -p consortium-builder

Running the full pipeline against the melt-pot examples rewrites their committed src/consortium.gen.rs. The embedded debug ELFs are named relative to the CSTI_MCU_PATH/CSTI_MANIFEST_PATH variables the builder injects into every endpoint build, so the file carries no build-host path.

License

Apache-2.0. See LICENSE.