<div align="center">
<img src="https://raw.githubusercontent.com/tracel-ai/xtask/main/assets/xtask.png" width="256px"/>
<h1>Tracel Xtask</h1>
[](https://discord.gg/uPEBbYYDB6)
[](https://crates.io/crates/tracel-xtask)
[](https://crates.io/crates/tracel-xtask)
[](https://github.com/tracel-ai/xtask/actions/workflows/ci.yml)
---
<br/>
</div>
A collection of easy-to-use and extensible commands to be used in your [xtask CLI][1] based on [clap][2].
We rely on these commands in each of our Tracel repositories. By centralizing our redundant commands we save a big amount
of code duplication, boilerplate and considerably lower their maintenance cost. This also provides a unified interface across
all of our repositories.
These commands are not specific to Tracel repositories and they should be pretty much usable in any Rust repositories with
a cargo workspace as well as other repositories where Rust is not necessarily the only language. The commands can be easily
extended using handy proc macros and by following some patterns described in this README.
## Getting Started
### Install xtask CLI
```bash
cargo install tracel-xtask-cli
```
### Setting Up a Cargo Workspace with an xtask binary crate
#### Using the CLI
**Not available yet**
#### Manually
1. **Create a new Cargo workspace:**
```bash
cargo new my_workspace
cd my_workspace
```
2. **Create the `xtask` binary crate:**
```bash
cargo new xtask --bin
```
3. **Configure the workspace:**
Edit the `Cargo.toml` file in the root of the workspace to include the following:
```toml
[workspace]
members = ["xtask"]
```
4. **Add the `tracel-xtask` dependency:**
In the `xtask/Cargo.toml` file, add the following under `[dependencies]`:
```toml
[dependencies]
tracel-xtask = "3.0"
```
5. **Build the workspace:**
```bash
cargo build
```
Your workspace is now set up with a `xtask` binary crate that depends on `tracel-xtask` version 2.0.x.
### Bootstrap main.rs
1. In the `main.rs` file of the newly created `xtask` crate, import the `tracel_xtask` prelude module and then declare
a `Command` enum. Select the base commands you want to use by adding the `macros::base_commands` attribute:
```rust
use tracel_xtask::prelude::*;
#[macros::base_commands(
Build,
Check,
Fix
Test,
)]
pub enum Command {}
```
2. Update the `main` function to initialize xtask and dispatch the base commands:
```rust
fn main() -> anyhow::Result<()> {
let args = init_xtask::<Command>(parse_args::<Command>()?)?;
match args.command {
// dispatch_base_commands function is generated by the commands macro
_ => dispatch_base_commands(args),
}
}
```
3. Build the workspace with `cargo build` at the root of the repository to verify that everything is.
4. You should now be able to display the main help screen which lists the commands you selected previously:
```sh
xtask --help
```
## Conventions
### Repository structure
All our repositories follow the same directory hierarchy:
- a `crates` directory which contains all the crates of the workspace
- an `examples` directory which holds all the examples crates
- a `xtask` directory is a release of xtask CLI with all the base commands
### About tests
As per Cargo convention, [Integration tests][3] are tests contained in a `tests` directory of a crate besides its `src` directory.
Inline tests in `src` directory are called [Unit tests][4].
`tracel-xtask` allows to easily execute them separately using the `test` command.
## Interface generalities
### Target
There are 4 default targets provided by `tracel-xtask`:
- `workspace` which targets the cargo workspace, this is the default target
- `crates` are all the binary crates and library crates
- `examples` are all the example crates
- `all-packages` are both `crates` and `examples` targets
`workspace` and `all-packages` are different because `workspace` uses the `--workspace` flag of cargo whereas `all-packages`
relies on `crates` and `examples` targets which use the `--package` flag. So `all-packages` executes a command for each crate
or example individually.
Here are some examples:
```sh
# run all the crates tests
xtask test --target crates all
# check format for examples, binaries and libs
xtask check --target all-packages unit
# build the workspace
xtask build --target workspace
# workspace is the default target so this has the same effect
xtask build
```
### Global options
The following options are global and precede the actual command on the command line.
#### Environment
`-e`, `--environment`
```sh
xtask -e production build
```
This `--environment` parameter is passed to all the `handle_command` function as `env`.
Its main role is to inform your custom commands or dispatch functions about the targeted environment. An environment
has 3 different display names: a short one, a medium one and a long one.
The available environments are listed below:
| Development | d | dev | development |
| Test | t | test | test |
| Staging | s | stag | staging |
| Production | p | prod | production |
It also automatically loads the following environment variables files if they exists in the working directory:
- `.env` for any set environment,
- `.env.secrets` containing secrets for any set environment,
- `.env.{environment}` (example: `.env.dev`) for the non-sensitive configuration,
- `.env.{environment}.secrets` (example `.env.dev.secrets`) for the sensitive configuration like password. These
files must be added to the ignore file of your VCS tool (for git add `.env.*.secrets` to the `.gitignore` file
at the root of your repository).
The `.secrets` component in `.env.secrets` is called a `dotenv family`. There exists two additional `dotenv families`:
- `.infra` for infrastructure related variable, this allows to nicely split application config from pure infra config,
- `.infra.secrets` for infrastructure secrets.
For each family both `.env.{family}` and `.env.{environment}.{family}` are sourced if they exists.
**Note:** If an environment variable is already defined in the process environment, it takes precedence over the
value defined in `.env` files.
At last, the display form used in dotenv filenames for the environment is the `medium` one (see table above).
#### Environment Index
By default the index is transparent because it is set to 1 and the first index does not appear in the environment name.
So for instance the `stag` environment is actually the `stag1` environment but it is always printed `stag`.
Any provided index greater than 1 is always explicit and always appear in the environment name.
To turn an environment into an explicit environment you can call the function `into_explicit` on the environment
which returns a new environment. This new environment always prints the index so instead of printing `s`, `stag`
or `staging`, it will print `s1`, `stag1` and `staging1` explicitly.
#### Context
`-c`, `--context`
```sh
xtask --context no-std build
```
This argument has no effect in the base commands. Its purpose is to provide additional context to your custom commands
or dispatch functions — for example, to indicate whether the current build context is `std` or `no-std`.
This parameter is passed to all the `handle_command` function as `context`.
#### Coverage
`--enable-coverage`
It setups the Rust toolchain to generate coverage information.
## Standard repository vs. Monorepo
`xtask` CLI is capable of handling traditional repositories with a single Rust workspase at the root of the repository and
monorepos which are a collection of Cargo workspaces in their own directories called `subrepos`.
A `subrepo` is basically the same as a standard repository with its own `Cargo.toml` and `xtask` crate. For instance this
simple monorepo contains the following files and directories at the root:
```
.git
.gitignore
Cargo.toml
[backend]
|__Cargo.toml
|__[crates]
|__[xtask]
[frontend]
|__Cargo.toml
|__[xtask]
```
`backend` and `frontend` are subrepos with separate Cargo workspaces with their own members. They all have an `xtask` member
to manage them.
The root `Cargo.toml` does not represent a real Cargo workspace, its only purpose is to centralize all the dependency versions
and features. `xtask` CLI will push the dependency versions in the root `Cargo.toml` to the subrepos `Cargo.toml` files
whenever a dependency is used in the subrepo.
For instance, say we have this root `Cargo.toml` file:
```toml
[workspace.dependencies]
log = "0.4.29"
rand = "0.9.2"
tokio = { version = "1.48.0", features = ["full"] }
```
and `backend` subrepo `Cargo.toml` is:
```toml
[workspace]
resolver = "2"
members = ["crates/*", "xtask"]
[workspace.dependencies]
rand = "0.8.0"
tokio = "1.46.0"
```
Whenever the `xtask` CLI executes a command it will enforce the dependency versions to by in sync with the root `Cargo.toml`,
which means that the `backendf` subrepo `Cargo.toml` will be modified to be:
```toml
[workspace]
resolver = "2"
members = ["crates/*", "xtask"]
[workspace.dependencies]
rand = "0.9.2"
tokio = { version = "1.48.0", features = ["full"] }
```
Note that it is possible to drop `features = ["full"]` from the root `Cargo.toml` file and make the decision about which
feature to use at the subrepo or even crate level.
## Anatomy of a base command
We use the derive API of clap which is based on structs, enums and attribute proc macros. Each base command is a
submodule of the `base_commands` module. If the command accepts arguments there is a corresponding struct named `<command>CmdArgs`
which declare the options, arguments and subcommands. In the case of subcommands a corresponding enum named `<command>SubCommand`
is defined.
Here is an example with a `foo` command:
```rust
#[macros::declare_command_args(Target, FooSubCommand)]
struct FooCmdArgs {}
pub enum FooSubCommand {
/// A sub command for foo (usage on the command line: xtask foo print-something)
PrintSomething,
}
```
Note that it is possible to have an arbitrary level of nested subcommands but deeper nested subcommands cannot be extended,
in other words, only the first level of subcommands can be extended. If possible, try to design commands with only one
level of subcommands to keep the interface simple.
In the following sections we will see how to create completely new commands as well how to extend existing base commands.
## Customization
The crate `xtask-tests` contains examples of the following sections.
### Create a new command
1. First, we organize commands by creating a `commands` module. Create a file `xtask/src/commands/mycommand.rs` as well
as the corresponding `mod.rs` file to declare the module contents.
2. Then, in `mycommand.rs` define the arguments struct with the `declare_command_args` macro and define the `handle_command`
function. The `declare_command_args` macro takes two parameters, the first is the type of the target enum and the second
is the type of the subcommand enum if any. If the command has no target or no subcommand then put `None` for each argument
respectively. `Target` is the default target type provided by `tracel-xtask`. This type can be extended to support more
targets as we will see in a later section.
```rust
use tracel_xtask::prelude::*;
#[macros::declare_command_args(Target, None)]
struct MyCommandCmdArgs {}
pub fn handle_command(_args: MyCommandCmdArgs) -> anyhow::Result<()> {
println!("Hello from my-command");
Ok(())
}
```
3. Make sure to update the `mod.rs` file to declare the command module:
```
pub(crate) mod my_command;
```
4. We can now add a new variant to the `Command` enum in `main.rs`:
```rust
mod commands;
use tracel_xtask::prelude::*;
#[macros::base_commands(
Bump,
Check,
Fix,
Test,
)]
pub enum Command {
MyCommand(commands::mycommand::MyCommandCmdArgs),
}
```
5. And dispatch its handling to our new command module:
```rust
fn main() -> anyhow::Result<()> {
let args = init_xtask::<Command>()?;
match args.command {
Command::NewCommand(args) => commands::new_command::handle_command(args),
_ => dispatch_base_commands(args),
}
}
```
6. You can now test your new command with:
```sh
xtask my-command --help
xtask my-command
```
### Extend the default Target enum
Let's implement a new command called `extended-target` to illustrate how to extend the default `Target` enum.
1. Create a `commands/extended_target.rs` file and update the `mod.rs` file as we saw in the previous section.
2. We also need to add a new `strum` dependency to our `Cargo.toml` file:
```toml
[dependencies]
strum = {version = "0.26.3", features = ["derive"]}
```
3. Then we can extend the `Target` enum with the `macros::extend_targets` attribute in our `extended_target.rs` file.
Here we choose to add a new target called `frontend` which targets the frontend component we could find for instance
in a monorepo:
```rust
use tracel_xtask::prelude::*;
#[macros::extend_targets]
pub enum MyTarget {
/// Target the frontend component of the monorepo.
Frontend,
}
```
4. Then we define our command arguments by referencing our newly created `MyTarget` enum in the `declare_command_args` attribute:
```rust
#[macros::declare_command_args(MyTarget, None)]
struct ExtendedTargetCmdArgs {}
```
5. Our new target is then available for use in the `handle_command` function:
```rust
pub fn handle_command(args: ExtendedTargetCmdArgs) -> anyhow::Result<()> {
match args.target {
// Default targets
MyTarget::AllPackages => println!("You chose the target: all-packages"),
MyTarget::Crates => println!("You chose the target: crates"),
MyTarget::Examples => println!("You chose the target: examples"),
MyTarget::Workspace => println!("You chose the target: workspace"),
// Additional target
MyTarget::Frontend => println!("You chose the target: frontend"),
};
Ok(())
}
```
6. Register our new command the usual way by adding it to our `Command` enum and dispatch it
in the `main` function:
```rust
mod commands;
use tracel_xtask::prelude::*;
#[macros::base_commands(
Bump,
Check,
Fix,
Test,
)]
pub enum Command {
ExtendedTarget(commands::extended_target::ExtendedTargetCmdArgs),
}
fn main() -> anyhow::Result<()> {
let args = init_xtask::<Command>()?;
match args.command {
Command::ExtendedTarget(args) => commands::extended_target::handle_command(args),
_ => dispatch_base_commands(args),
}
}
```
7. Test the command with:
```rust
xtask extended-target --help
xtask extended-target --target frontend
```
### Target aliases
When extending a command we can mark a new variant as an _alias_ of base target (the default ones).
This example defines `Backend` target as the same as `Worspace` target for all the base commands:
```rust
use tracel_xtask::prelude::*;
#[macros::extend_targets]
pub enum MyTarget {
/// Target the backend component of the monorepo, same thing as Workspace
#[alias(Workspace)]
Backend,
/// Target the frontend component of the monorepo.
Frontend,
}
```
### Extend a base command
To extend an existing command we use the `macros::extend_command_args` attribute which takes three parameters:
- first argument is the type of the base command arguments struct to extend,
- second argument is the target type (or `None` if there is no target),
- third argument is the subcommand type (or `None` if there is no subcommand).
Let's use two examples to illustrate this, the first is a command to extend the `build` base command with
a new `--debug` argument; and the second is a new command to extend the subcommands of the `check` base command
to add a new `my-check` subcommand.
Note that you can find more examples in the `xtask` crate of this repository.
#### Extend the arguments of a base command
We create a new command called `extended-build-args` which will have an additional argument called `--debug`.
1. Create the `commands/extended_build_args.rs` file and update the `mod.rs` file as we saw in the previous section.
2. Extend the `BuildCommandArgs` struct using the attribute `macros::extend_command_args` and define the `handle_command` function.
Note that the macro automatically implements the `TryInto` trait which makes it easy to dispatch back to the base command
own `handle_command` function. Also note that if the base command requires a target then you need to provide a target as well
in your extension, i.e. the target parameter of the macro cannot be `None` if the base command has a `Target`.
```rust
use tracel_xtask::prelude::*;
#[macros::extend_command_args(BuildCmdArgs, Target, None)]
pub struct ExtendedBuildArgsCmdArgs {
/// Print additional debug info when set
#[arg(short, long)]
pub debug: bool,
}
pub fn handle_command(args: ExtendedBuildArgsCmdArgs) -> anyhow::Result<()> {
if args.debug {
println!("Debug is enabled");
}
base_commands::build::handle_command(args.try_into().unwrap())
}
```
3. Register the new command the usual way by adding it to the `Command` enum and dispatch it
in the `main` function:
```rust
mod commands;
use tracel_xtask::prelude::*;
#[macros::base_commands(
Bump,
Check,
Fix,
Test,
)]
pub enum Command {
ExtendedBuildArgs(commands::extended_build_args::ExtendedBuildArgsCmdArgs),
}
fn main() -> anyhow::Result<()> {
let args = init_xtask::<Command>()?;
match args.command {
Command::ExtendedBuildArgs(args) => commands::extended_build_args::handle_command(args),
_ => dispatch_base_commands(args),
}
}
```
4. Test the command with:
```rust
xtask extended-build-args --help
xtask extended-build-args --debug
```
#### Extend the subcommands of a base command
For this one we create a new command called `extended-check-subcommands` which will have an additional subcommand.
1. Create a `commands/extended_check_subcommands.rs` file and update the `mod.rs` file as we saw in the previous section.
2. Extend the `CheckCommandArgs` struct using the attribute `macros::extend_command_args`:
```rust
use tracel_xtask::prelude::*;
#[macros::extend_command_args(CheckCmdArgs, Target, ExtendedCheckSubcommand)]
pub struct ExtendedCheckedArgsCmdArgs {}
```
3. Implement the `ExtendedCheckSubcommand` enum by extending the `CheckSubcommand` base enum with the macro `extend_subcommands`.
It takes the name of the type of the subcommand enum to extend:
```rust
#[macros::extend_subcommands(CheckSubCommand)]
pub enum ExtendedCheckSubcommand {
/// An additional subcommand for our extended check command.
MySubcommand,
}
```
4. Implement the `handle_command` function to handle the new subcommand. Note that we must handle the `All` subcommand as well:
```rust
use strum::IntoEnumIterator;
pub fn handle_command(args: ExtendedCheckedArgsCmdArgs) -> anyhow::Result<()> {
match args.get_command() {
ExtendedCheckSubcommand::MySubcommand => run_my_subcommand(args.clone()),
ExtendedCheckSubcommand::All => {
ExtendedCheckSubcommand::iter()
.filter(|c| *c != ExtendedCheckSubcommand::All)
.try_for_each(|c| {
handle_command(
ExtendedCheckedArgsCmdArgs {
command: Some(c),
target: args.target.clone(),
exclude: args.exclude.clone(),
only: args.only.clone(),
},
)
})
}
_ => base_commands::check::handle_command(args.try_into().unwrap()),
}
}
fn run_my_subcommand(_args: ExtendedCheckedArgsCmdArgs) -> Result<(), anyhow::Error> {
println!("Executing new subcommand");
Ok(())
}
```
5. Register the new command the usual way by adding it to the `Command` enum and dispatch it
in the `main` function:
```rust
mod commands;
use tracel_xtask::prelude::*;
#[macros::base_commands(
Bump,
Check,
Fix,
Test,
)]
pub enum Command {
ExtendedCheckSubcommand(commands::extended_check_subcommands::ExtendedCheckedArgsCmdArgs),
}
fn main() -> anyhow::Result<()> {
let args = init_xtask::<Command>()?;
match args.command {
Command::ExtendedCheckSubcommand(args) => commands::extended_check_subcommands::handle_command(args),
_ => dispatch_base_commands(args),
}
}
```
6. Test the command with:
```rust
xtask extended-check-subcommands --help
xtask extended-check-subcommands my-check
```
## Custom builds and tests
`tracel-xtask` provides helper functions to easily execute custom builds or tests with specific features or build targets (do not confuse
Rust build targets which is an argument of the `cargo build` command with the xtask target we introduced previously).
For instance we can extend the `build` command to build additional crates with custom features or build targets using the helper function:
```rust
pub fn handle_command(mut args: tracel_xtask::commands::build::BuildCmdArgs) -> anyhow::Result<()> {
// regular execution of the build command
tracel_xtask::commands::build::handle_command(args)?;
// additional crate builds
// build 'my-crate' with all the features
tracel_xtask::utils::helpers::custom_crates_build(vec!["my-crate"], vec!["--all-features"], None, None, "all features")?;
// build 'my-crate' with specific features
tracel_xtask::utils::helpers::custom_crates_build(vec!["my-crate"], vec!["--features", "myfeature1,myfeature2"], None, None, "myfeature1,myfeature2")?;
// build 'my-crate' with a different target than the default one
tracel_xtask::utils::helpers::custom_crates_build(vec!["my-crate"], vec!["--target", "thumbv7m-none-eabi"], None, None, "thumbv7m-none-eabi target")?;
Ok(())
}
```
## Enable and generate coverage information
Here is a example GitHub job which shows how to setup coverage, enable it and upload coverage information to codecov:
```yaml
env:
GRCOV_LINK: "https://github.com/mozilla/grcov/releases/download"
GRCOV_VERSION: "0.8.19"
jobs:
my-job:
runs-on: ubuntu-22.04
steps:
- name: Checkout
uses: actions/checkout@v4
- name: install rust
uses: dtolnay/rust-toolchain@master
with:
components: rustfmt, clippy
toolchain: stable
- name: Install grcov
shell: bash
run: |
curl -L "$GRCOV_LINK/v$GRCOV_VERSION/grcov-x86_64-unknown-linux-musl.tar.bz2" |
tar xj -C $HOME/.cargo/bin
xtask coverage install
- name: Build
shell: bash
run: xtask build
- name: Tests
shell: bash
run: xtask --enable-coverage test all
- name: Generate lcov.info
shell: bash
# /* is to exclude std library code coverage from analysis
run: xtask coverage generate --ignore "/*,xtask/*,examples/*"
- name: Codecov upload lcov.info
uses: codecov/codecov-action@v4
with:
files: lcov.info
token: ${{ secrets.CODECOV_TOKEN }}
```
## Special command 'validate'
By convention this command is responsible to run all the checks, builds, and/or tests that validate the code
before opening a pull request or merge request.
The command `Validate` can been added via the macro `tracel_xtask_macros::commands` like the other commands.
By default all the checks from the `check` command are run as well as both unit and integration tests from
the `test` command.
You can make your own `handle_command` function if you need to perform more validations. Ideally this function
should only call the other commands `handle_command` functions.
For quick reference here is a simple example to perform all checks and tests against the workspace:
```rust
pub fn handle_command(args: ValidateCmdArgs) -> anyhow::Result<()> {
let target = Target::Workspace;
let exclude = vec![];
let only = vec![];
// checks
[
CheckSubCommand::Audit,
CheckSubCommand::Format,
CheckSubCommand::Lint,
CheckSubCommand::Typos,
]
.iter()
.try_for_each(|c| {
super::check::handle_command(CheckCmdArgs {
target: target.clone(),
exclude: exclude.clone(),
only: only.clone(),
command: Some(c.clone()),
ignore_audit: args.ignore_audit,
})
})?;
// tests
super::test::handle_command(TestCmdArgs {
target: target.clone(),
exclude: exclude.clone(),
only: only.clone(),
threads: None,
jobs: None,
command: Some(TestSubCommand::All),
})?;
Ok(())
}
```
## Base commands list
### Check and Fix
The `check` and `fix` commands are designed to help you maintain code quality during development.
They run various checks and fix issues, ensuring that your code is clean and follows best practices.
`check` and `fix` contains the same subcommands to audit, format, lint or proofread a code base.
While the `check` command only reports issues, the `fix` command attempts to fix them as they are encountered.
Each check can be executed separately or all of them can be executed sequentially using `all`.
Usage to lint the code base:
```sh
xtask check lint
xtask fix lint
xtask fix all
```
### Running Tests
Testing is a crucial part of development, and the `test` command is designed to make this process easy.
This command makes the distinction between unit tests and integrations tests. [Unit tests][4] are inline tests under the
`src` directory of a crate. [Integration tests][3] are tests defined in files under the `tests` directory of a crate besides
the `src` directory.
Usage:
```sh
# execute workspace unit tests
xtask test unit
# execute workspace integration tests
xtask test integration
# execute workspace both unit tests and integration tests
xtask test all
```
Note that documentation tests are supported by the `doc` command.
### Documentation
Command to build and test the documentation in a workspace.
### Bumping Versions
This is a command reserved for repository maintainers.
The `bump` command is used to update the version numbers of all first-party crates in the repository.
This is particularly useful when you're preparing for a new release and need to ensure that all crates have the correct version.
You can bump the version by major, minor, or patch levels, depending on the changes made.
For example, if you’ve made breaking changes, you should bump the major version.
For new features that are backwards compatible, bump the minor version.
For bug fixes, bump the patch version.
Usage:
```sh
xtask bump <SUBCOMMAND>
```
### Publishing Crates
This is a command reserved for repository maintainers and is typically used in `publish` GitHub workflows.
This command automates the process of publishing crates to `crates.io`, the Rust package registry.
By specifying the name of the crate, `xtask` handles the publication process, ensuring that the crate is available for others to use.
Usage:
```sh
xtask publish <NAME>
```
As mentioned, this command is often used in a GitHub workflow.
We provide a Tracel's reusable [publish-crate][8] workflow that makes use of this command.
Here is a simple example with a workflow that publishes two crates A and B with A depending on B.
```yaml
name: publish all crates
on:
push:
tags:
- "v*"
jobs:
publish-B:
uses: tracel-ai/github-actions/.github/workflows/publish-crate.yml@v6
with:
crate: B
secrets:
CRATES_IO_API_TOKEN: ${{ secrets.CRATES_IO_API_TOKEN }}
# --------------------------------------------------------------------------------
publish-A:
uses: tracel-ai/github-actions/.github/workflows/publish-crate.yml@v6
with:
crate: A
needs:
- publish-B
secrets:
CRATES_IO_API_TOKEN: ${{ secrets.CRATES_IO_API_TOKEN }}
```
### Coverage
This command provide a subcommand to install the necessary dependencies for performing code coverage and a subcommand to generate the
coverage info file that can then be uploaded to a service provider like codecov. See dedicated section `Enable and generate coverage information`.
### Container
The `container` command standardizes how we build, publish, promote, and roll out containers.
It currently supports **Docker** for container builds, **AWS ECR** as the container registry and **AWS EC2 Auto Scaling Groups (ASG)** for deployment rollouts.
The design, however, could later be refactored to support other cloud providers or services (such as podman, AWS ECS, GCP Artifact Registry, Azure Container Registry, or self-hosted registries) with ideally minimal changes to the workflow.
#### Conceptual Model
##### Deploy
**Build of container** -> **Push to container registry** -> **Promote to `<env_medium>`** -> **Rollout**
##### Rollback
**Rollback by promoting `rollback_<env_medium>` to `<env_medium>`** -> **Rollout**
#### Prerequisites
- **AWS CLI** setup with permissions for ECR and Auto Scaling.
- **Docker** installed locally or on your CI runners.
#### Examples
First build the container locally:
```
xtask -e stag container build --allow-dirty --context-dir . --build-file Dockerfile --image my_image
```
Retrieve the commit `SHA` tag from the output of the build command and push the container to the registry:
```
xtask -e stag container push --image my_image --local-tag SHA --repository my_image --auto-remote_tag --region AWS_REGION
```
The container image will be pushed with the commit SHA tag and a monotonic number tag.
Promote the pushed image to mark it as latest (move the latest tag to it):
```
xtask -e stag container promote --repository my_image --tag SHA --region AWS_REGION
```
Then to actually deploy to the targeted environment, initiate a rollout:
```
xtask -e stag container rollout --region AWS_REGION --asg ASG_NAME --wait
```
To see the current `latest` and `rollback` images use the `list` subcommand:
```
xtask -e stag container list --region AWS_REGION --repository my_image
```
### Secrets
This command handles secrets to view, edit and even write an env file with them. For now it only supports AWS Secrets Manager.
#### Examples
View secrets:
```
xtask -e stag secrets view --region AWS_REGION my_secret
```
### Docker Compose
The `docker-compose` command provides `up` and `down` commands to start and stop stacks. The command is integrated with the environment
configuration mechanism of `tracel-xtask`.
The name of the compose file must follow the template `docker-compose.{env}.yml` with `env` being the shorthand environment name.
For instance for the development environment the file is named `docker-compose.dev.yml`.
The command also requires a mandatory project name for the stack in order to have idempotent `up` commands.
### Dependencies
Various additional subcommands about dependencies.
`deny` make sure that all dependencies meet requirements using [cargo-deny][5].
`unused` detects dependencies in the workspace that are not in ussed.
### Vulnerabilities
This command makes it easier to execute sanitizers as described in [the Rust unstable book][6].
These sanitizers require a nightly toolchain.
```
Run the specified vulnerability check locally. These commands must be called with 'cargo +nightly'
Usage: xtask vulnerabilities <COMMAND>
Commands:
all Run all most useful vulnerability checks
address-sanitizer Run Address sanitizer (memory error detector)
control-flow-integrity Run LLVM Control Flow Integrity (CFI) (provides forward-edge control flow protection)
hw-address-sanitizer Run newer variant of Address sanitizer (memory error detector similar to AddressSanitizer, but based on partial hardware assistance)
kernel-control-flow-integrity Run Kernel LLVM Control Flow Integrity (KCFI) (provides forward-edge control flow protection for operating systems kerneljs)
leak-sanitizer Run Leak sanitizer (run-time memory leak detector)
memory-sanitizer Run memory sanitizer (detector of uninitialized reads)
mem-tag-sanitizer Run another address sanitizer (like AddressSanitizer and HardwareAddressSanitizer but with lower overhead suitable for use as hardening for production binaries)
nightly-checks Run nightly-only checks through cargo-careful `<https://crates.io/crates/cargo-careful>`
safe-stack Run SafeStack check (provides backward-edge control flow protection by separating stack into safe and unsafe regions)
shadow-call-stack Run ShadowCall check (provides backward-edge control flow protection - aarch64 only)
thread-sanitizer Run Thread sanitizer (data race detector)
help Print this message or the help of the given subcommand(s)
```
## Utilities
### Easy CTRL+c management
`tracel-xtask` gives access to two useful macros `register_cleanup` and `handle_cleanup` to easily define some cleanup functions to be executed at
a given time during the program as well as whenever the user presses <kbd>CTRL+c</kbd>.
It is very useful to guard processes while executing some tests and make sure that the state is still cleaned up even if the program is interrupted by
the user.
Since the cleanup handler is stored in a static variable, it's `drop` function is not automatically called when the program exits normally and the `handle_cleanup` macro needs to be called manually. However, when the program is interrupted by the user with <kbd>CTRL+c</kbd>, the cleanup is automatically called.
Example:
Register cleanup functions in your commands, say you have a customized `test` command that spins up some container.
```rs
pub(crate) async fn handle_command(
args: TestCmdArgs,
env: Environment,
ctx: Context,
) -> anyhow::Result<()> {
match args.get_command() {
TestSubCommand::Integration => {
// spin up containers
// ...
// register cleanup command for them
register_cleanup!("Integration tests: Docker compose stack", move || {
base_commands::docker::handle_command(
DockerCmdArgs {
build: false,
project: super::DOCKER_COMPOSE_PROJECT_NAME.to_string(),
command: Some(DockerSubCommand::Down),
services: vec![],
},
Environment::Test,
ctx.clone(),
)
.expect("Should be able to stop docker compose stack");
});
// Execute xtask test base command
base_commands::test::run_integration(&cmd_args.target, &cmd_args)
}
TestSubCommand::All => { ... }
}
}
```
Then call the `handle_cleanup` macro at the end of your main function to force a cleanup:
```rs
use tracel_xtask::prelude::*;
#[macros::base_commands(
Build,
Check,
Fix,
Test
)]
pub enum Command {}
fn main() -> anyhow::Result<()> {
let args = init_xtask::<Command>(parse_args::<Command>()?)?;
match args.command {
Command::Test(cmd_args) => {
commands::test::handle_command(cmd_args, args.environment, args.context).await
}
_ => dispatch_base_commands(args),
}
handle_cleanup!();
}
```
[1]: https://github.com/matklad/cargo-xtask
[2]: https://github.com/clap-rs/clap
[3]: https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html
[4]: https://doc.rust-lang.org/book/ch11-03-test-organization.html#integration-tests
[5]: https://doc.rust-lang.org/book/ch11-03-test-organization.html#unit-tests
[6]: https://embarkstudios.github.io/cargo-deny/
[7]: https://doc.rust-lang.org/beta/unstable-book/compiler-flags/sanitizer.html
[8]: https://github.com/tracel-ai/github-actions/blob/main/.github/workflows/publish-crate.yml