# consortium-builder
Compiles, post-processes, and stages every artifact a `Consortium.toml` describes.
[`consortium-cfg`](../consortium-cfg) answers _"what does this system look like, and is
it valid?"_ and emits generated source. This crate answers _"now actually build it"_:
four compilers, a post-processing step remoteproc requires, and a staging tree a
packaging recipe can consume.
It stops short of deployment. **Nothing here writes to a live target filesystem** — the
output is a local `dist/` tree plus an optional BitBake hand-off.
The user-facing entry point is [`csti build`](../consortium-cli); this crate is the
library behind it.
## Pipeline
```text
Consortium.toml
├─ 1. firmware cargo, bare-metal target → ELF → rename .vector_table
├─ 2. elf_map the defmt map the app embeds for decoding
├─ 3. hmi vp / npm / bun → frontend bundle
├─ 4. tee OP-TEE TA dev kit → signed <uuid>.ta
├─ 5. application cargo, usually cross-compiled → Linux executable
├─ 6. systemd render the unit
├─ 7. dist assemble dist/ + a BitBake recipe
└─ 8. bitbake optional stage / invoke / dry-run hand-off
```
### Plan, then run
`pipeline::plan` resolves every stage's inputs — paths, env vars, artifact locations —
**without running any build**, threading the data flow between stages. `pipeline::run`
plans, then executes.
That split is what makes the pipeline debuggable. A wrong sysroot or a missing firmware
ELF surfaces as a planning failure naming the stage, rather than as a cargo error
several minutes into a cross-compile.
## The `Compiler` trait
Firmware, the Linux application, the OP-TEE TA, and the HMI frontend are built by very
different toolchains:
| `firmware` | coprocessor ELF | `cargo`, bare-metal target |
| `application` | Linux executable | `cargo`, usually cross-compiled |
| `optee` | signed OP-TEE TA | the OP-TEE TA dev kit |
| `web` | HMI frontend bundle | `vp` / `npm` (Cog) or `bun` (Pocket) |
`Compiler` makes them uniform: each implementor owns a `Configuration` carrying all its
parameters, a typed `Error`, and an `Output`. Its methods take the config rather than
`&self`, so every implementor has the same `.build(cfg)` / `.output_path(&cfg)` shape
and the pipeline needs no per-kind special casing.
## Compile-time configuration
Memory layout lives in generated source, not the environment. Controller firmware gets
its IPC region base and its `[dbg]` ring baked into generated `init()`, and the Linux
application reaches the same memory through its UIO devices, so there are no
`CONSORTIUM_IPC_*` or `CONSORTIUM_DBG_*` variables.
What still arrives as environment variables is what generated code cannot know, read
with `option_env!`/`env!`: the on-target HMI asset paths (`compile::env::HmiConfig`), the
TEE UUID, and the `CSTI_*` build-host directories (`CstiWorkspace`). Numeric values are
emitted as decimal strings.
## Things that are easy to get wrong
**Firmware post-processing is mandatory.** Linux remoteproc expects a coprocessor's
vector table in a section named `.isr_vectors`, but `cortex-m-rt` emits
`.vector_table`. The pipeline renames it and keeps the original alongside the
deployable ELF — an ELF that boots and an ELF a debugger understands are not quite the
same file.
**Sysroot resolution has an order.** `[profile] sysroot` first, then
`SDKTARGETSYSROOT` / `OECORE_TARGET_SYSROOT`, so a manifest can pin a sysroot without
the developer having sourced an SDK script. On an arm64 Linux host,
`aarch64-unknown-linux-gnu` is a _native_ build and `sysroot = "/"` resolves to the
image root — which is how the melt-pot manifests build inside an Apple `container` VM
with no SDK at all.
**TA signing is optional on purpose.** Signing needs the dev kit's signer script and
keys, which a laptop or CI runner usually lacks, so the unsigned ELF can be staged
instead (`--no-sign-ta`). Compiling the TA and signing it are separable steps.
**Linux device-tree sources are external inputs.** `csti build --linux <DIR>`
selects the BSP `.dts`/`.dtsi`, follows its local includes, and supplies
`dt-bindings/*.h` for `dtc`. Consortium bundles none of those Linux sources.
**The HMI build command can be overridden.** `CONSORTIUM_HMI_BUILD_CMD` takes
precedence over `vp`/`npm`. That exists because the frontend sometimes has to be built
elsewhere: inside a Linux container the host's `node_modules/` is already populated, so
pointing the command at `/bin/true` lets the pipeline consume the existing dist.
## Staging
The assembled tree can contain `app/`, `firmware/`, `ta/`, `www/`, `dts/linux/`,
`kernel/`, `boot/`, and `systemd/`, plus a generated BitBake recipe.
Firmware ELFs land at `dist/firmware/<core>.elf` so a firmware-packaging recipe can
install them to `/lib/firmware/<core>` for remoteproc's kernel-side loader. That is
unrelated to the defmt ELF map, which is embedded in the application binary.
`[profile] deploy_root` is a **naming convention only** — it composes the HMI dist
path, the systemd unit's `ExecStart=`/`WorkingDirectory=`, and the firmware install
location a recipe consumes. The pipeline never writes there.
## BitBake hand-off
Staging (`--bitbake-files-dir`) and invoking (`--bitbake-recipe` +
`--bitbake-build-dir`) compose: given both, `dist/` is staged into the recipe's
`files/` directory first, then BitBake runs. `--bitbake-dry-run` records the command
without running it — that is the split-machine flow, where `dist/` is produced on one
host and the Yocto build happens on another.
## Extending
Prefer extending `consortium-cfg` lowering first, then this pipeline. When working
here, keep the four boundaries distinct: **planning**, **compilation**, **staging**,
and the **external BitBake hand-off**. Mixing them is what makes a build pipeline
untestable.
`progress::BuildReporter` is deliberately minimal — library callers can ignore it and
keep inherited subprocess output, while the CLI passes a reporter that holds a
persistent step prompt under cargo/vp/bitbake output.
## Testing
```bash
cargo test -p consortium-builder
```
Running the full pipeline against the melt-pot examples rewrites their committed
`src/consortium.gen.rs`. The embedded debug ELFs are named relative to the
`CSTI_MCU_PATH`/`CSTI_MANIFEST_PATH` variables the builder injects into every endpoint
build, so the file carries no build-host path.
## License
Apache-2.0. See [LICENSE](../../LICENSE).