hakari

hakari is a set of tools to manage workspace-hack packages.

Examples

use guppy::MetadataCommand;
use hakari::{HakariBuilder, TomlOptions};

// Use this workspace's PackageGraph for these tests.
let package_graph = MetadataCommand::new()
    .build_graph()
    .expect("obtained cargo-guppy's PackageGraph");
// The second argument to HakariBuilder::new specifies a Hakari (workspace-hack) package. At
// the moment cargo-guppy does not have such a package, and it is a TODO to add one.
let hakari_builder = HakariBuilder::new(&package_graph, None)
    .expect("HakariBuilder was constructed");

// HakariBuilder has a number of config options. For this example, use the defaults.
let hakari = hakari_builder.compute();

// "hakari" can be used to build a TOML representation that forms part of a Cargo.toml file.
// Existing Cargo.toml files can be managed using Hakari::read_toml.
let toml = hakari.to_toml_string(&TomlOptions::default()).expect("TOML output was constructed");

// toml contains the Cargo.toml [dependencies] that would go in the Hakari package. It can be
// written out through `HakariCargoToml` (returned by Hakari::read_toml) or manually.
println!("Cargo.toml contents:\n{}", toml);

The cargo-guppy repository also has a number of fixtures that demonstrate Hakari's output. Here is an example.

What are workspace-hack packages?

Let's say you have a Rust crate my-crate with two dependencies:

# my-crate/Cargo.toml
[dependencies]
foo = "1.0"
bar = "2.0"

Let's say that foo and bar both depend on baz:

# foo-1.0/Cargo.toml
[dependencies]
baz = { version = "1", features = ["a", "b"] }

# bar-2.0/Cargo.toml
[dependencies]
baz = { version = "1", features = ["b", "c"] }

What features is baz built with?

One way to resolve this question might be to build baz twice with each requested set of features. But this is likely to cause a combinatorial explosion of crates to build, so Cargo doesn't do that. Instead, Cargo builds baz once with the union of the features enabled for the package: [a, b, c].

NOTE: This description elides some details around unifying build and dev-dependencies: for more about this, see the documentation for guppy's CargoResolverVersion.

Now let's say you're in a workspace, with a second crate your-crate:

# your-crate/Cargo.toml
[dependencies]
baz = { version = "1", features = ["c", "d"] }

In this situation:

if you build	`baz` is built with
just `my-crate`	`a, b, c`
just `your-crate`	`c, d`
`my-crate` and `your-crate` at the same time	`a, b, c, d`

Even in this simplified scenario, there are three separate ways to build baz. For a dependency like syn that have many optional features, large workspaces end up with a very large number of possible build configurations.

Even worse, the feature set of a package affects everything that depends on it, so syn being built with a slightly different feature set than before would cause *every package that directly or transitively depends on syn to be rebuilt. For large workspaces, this can result a lot of wasted build time.

To avoid this problem, many large workspaces contain a workspace-hack package. The purpose of this package is to ensure that dependencies like syn are always built with the same feature set no matter which workspace packages are currently being built. This is done by:

adding dependencies like syn to workspace-hack with the full feature set required by any package in the workspace
adding workspace-hack as a dependency of every crate in the repository.

Some examples of workspace-hack packages:

Rust's rustc-workspace-hack
Firefox's mozilla-central-workspace-hack
Diem's diem-workspace-hack

These packages have historically been maintained by hand, on a best-effort basis. hakari is an attempt to automate the maintenance of these packages.

How `hakari` works

Hakari follows a three-step process.

1. Configuration

A HakariBuilder provides options to configure how a Hakari computation is done. Options supported include:

With the optional summaries feature, HakariBuilder options can be read from or written to a file as TOML or some other format.

2. Computation

Once a HakariBuilder is configured, its compute method can be called to create a Hakari instance. The algorithm runs in three steps:

Use guppy to simulate a Cargo build for every workspace package and every given platform, with no features, default features and all features. Collect the results into a map indexed by every dependency and the different sets of features it was built with.
Scan through the map to figure out which dependencies are built with two or more different feature sets, collecting them into an output map.
If one assumes that the output map will be written out to the workspace-hack package through step 3 below, it is possible that it causes some extra packages to be built with a second feature set. Look for such packages, add them to the output map, and iterate until a fixpoint is reached and no new packages are built more than one way.

This computation is done in a parallel fashion, using the Rayon library.

The result of this computation is a Hakari instance.

3. Serialization

The last step is to serialize the contents of the output map into the workspace-hack package's Cargo.toml file.

[Hakari::read_toml] reads an existing Cargo.toml file on disk. This file is partially generated:
```
[package]
name = "workspace-hack"
version = "0.1.0"
# more options...

### BEGIN HAKARI SECTION
...
### END HAKARI SECTION
```
The contents outside the BEGIN HAKARI SECTION and END HAKARI SECTION lines may be edited by hand. The contents within this section are automatically generated.

On success, a HakariCargoToml is returned.
Hakari::to_toml_string returns the new contents of the automatically generated section.
HakariCargoToml::write_to_file writes out the contents to disk.

HakariCargoToml also supports serializing contents to memory and producing diffs.

Future work

hakari is a work-in-progress and is still missing many core features:

Simulating cross-compilations
Omitting some packages on some environments
Excluding some packages from the final result
Only including a subset of packages in the final result (e.g. unifying core packages like syn but not any others)
Automating the creation of workspace-hack packages
Support for alternate registries (depends on Cargo issue #9052)
A command-line interface

These features will be added as time permits.

Contributing

See the CONTRIBUTING file for how to help out.

License

This project is available under the terms of either the Apache 2.0 license or the MIT license.

hakari 0.1.0