corgi-build 0.1.38

PoC: deterministic, content-addressed, lock-free cargo-compatible build tool
corgi-build-0.1.38 is not a library.

corgi

A (mostly) cargo compatible build tool for Rust that shares the cache between multiple worktrees with no coarse-grained locking. This lets you make progress in all your worktrees concurrently, and saves significant energy, time and disk space.

While the original idea was to make builds fully reproducible, rustc's approach to incremental compilation shattered our dreams (and we need to use that to be as fast as cargo).

However, in the spirit of that original goal, we do still run build.rs and proc macros in a sandbox. (it disallows access to the network, file system and environment). This allows much tighter control of the inputs to the build process and improves caching (and as a side effect makes it a bit more difficult for supply chain attackes to work - assuming you never run the code you compile of course :D)

We copy best practices from elsewhere in the ecosystem. Feature flags are unified by default as would be done by workspace-hack so that changing the compilation root doesn't destroy your cache. Tests are run fully in parallel (like nextest), and we cache full-package test runs if they succeed. Cross-platform builds (currently macOS host building Linux binaries) use zigbuild under the hood, but hook into the existing cache.

Corgi does not attempt to completely replace cargo. We still use it for dependency and lockfile management. It is designed to be run side-by-side so you can adopt corgi on your machine without needing to mess with CI (or your colleagues).

We intend to (but haven't yet) build support for Linux and Windows; and also to provide a mechanism for trusted team-mates to share cached builds between machines.

Usage

cargo install corgi-build

corgi [ build | bench | run | test | check | clippy | fmt | audit | clean ] ...

Benchmarks use the same target declarations as Cargo. A benchmark using Rust's built-in harness needs no additional configuration:

[[bench]]
name = "built_in"

A benchmark framework that provides its own main, such as Criterion, disables the built-in harness:

[[bench]]
name = "criterion"
harness = false

Both forms can be selected when checking or running:

corgi check --bench criterion
corgi bench --bench criterion -- --sample-size 50

There are definitely sub-options and commands missing; please file reports or send PRs.

Migrating

Corgi already works to compile Zed and Delta, and a bunch of other Zed code.

If you aren't doing anything fancy in build.rs or proc macros, then you'll be fine (most ecosystem crates, with the exception of scratch, seem to fall into this category).

That said:

  • corgi requires a rust-toolchain.toml to pin the rust version exactly.
  • corgi disallows network access in build.rs and proc-macros. You can configure downloads in the corgi.toml and may need to update your build scripts.
  • corgi disallows reading/writing arbitrary file-system paths. You can opt-in to reading more files in corgi.toml.
  • corgi does not inherit the passive shell env (though again you can opt-in).
  • on macOS a binary's debug info lives in object files that corgi keeps in the cache rather than in target/, so lldb target/debug/whatever works from anywhere, until the objects are trimmed for going five days unused (rebuilding restores them).
  • corgi disallows you to write the CARGO_MANIFEST_DIR/CARGO_MANIFEST_PATH into the resulting binary (but we still do set them because there are just too many random crates out there that use them).

That said, corgi is very early; and you may run into issues I didn't yet, in which case reach out and we'll figure it out (or send a PR).

AI disclaimer

Almost all the code in the repository was generated by LLMs (though I wrote the README manually).