parabuild-rust

Parabuild is a Rust tool that helps you compile complex (single file) projects in parallel, such as some C++/CUDA projects that heavily use templates (when you cannot achieve the best performance through make -j).

Quick Start

The following is an example of how to use parabuild-rust to compile a C++ project.

We use handlebars templating language to generate source file, here is an example:

#include <iostream>

template <int n>
void print()
{
    std::cout << n << std::endl;
}

int main()
{
    print<{{N}}>();
    return 0;
}

Main body:

use parabuild::Parabuilder;
use serde_json::{json, to_string_pretty, Value as JsonValue};

fn main() {
    let project_path = "tests/example_cmake_project"; // your project path
    let workspaces_path = "workspaces"; // where to store the workspaces, executables, etc.
    let template_path = "src/main.cpp.template"; // template file in the project
    let target_executable_file = "build/main"; // target executable file
    let datas = vec![json!({"N": "10"}), json!({"N": "20"})];
    let mut parabuilder = Parabuilder::new(
        project_path,
        workspaces_path,
        template_path,
        &[target_executable_file],
    );
    parabuilder.set_datas(datas).unwrap();
    parabuilder.init_workspace().unwrap();
    let (run_data, _compile_error_datas): (JsonValue, Vec<JsonValue>) = parabuilder.run().unwrap();
    println!("{}", to_string_pretty(&run_data).unwrap());
    /*
    [
        {
            "data": {
                "N": "10"
            },
            "status": 0,
            "stderr": "",
            "stdout": "10\n"
        },
        {
            "data": {
                "N": "20"
            },
            "status": 0,
            "stderr": "",
            "stdout": "20\n"
        }
    ]
     */
}

We return compute_error_datas to indicate the data with compilation errors. Compilation errors are common in debugging projects that heavily use templates.

Advanced Usage

For more advanced usage, please refer to the documentation and complete example.

Command Line

We also provide a command line tool to compile the project. You can use cargo install parabuild to install it.

Simple Example

parabuild \
    tests/example_cmake_project \
    src/main.cpp \
    build/main \
    --in-place-template \
    --data '[{"N": 10}, {"N": 20}]'

Help

$ parabuild --help
A parallel build utility for template heavy projects.

Usage: parabuild [OPTIONS] <PROJECT_PATH> <TEMPLATE_FILE> [TARGET_FILES]...

Arguments:
  <PROJECT_PATH>     project path
  <TEMPLATE_FILE>    template file in the project
  [TARGET_FILES]...  target files in the project, which will be moved between build/run workspaces for further processing

Options:
  -w, --workspaces-path <WORKSPACES_PATH>
          where to store the workspaces, executables, etc [default: workspaces]
      --data <DATA>
          json format data
  -d, --data-file <DATA_FILE>
          json format data file, when used together with the `--data` option, ignore this option
  -o, --output-file <OUTPUT_FILE>
          output the json format result to a file, default to stdout
      --init-bash-script-file <INIT_BASH_SCRIPT_FILE>
          init bash script file
  -i, --init-cmake-args <INIT_CMAKE_ARGS>
          init cmake args
      --compile-bash-script-file <COMPILE_BASH_SCRIPT_FILE>
          compile bash script file
  -m, --make-target <MAKE_TARGET>
          make target, when used together with the `--compile-bash-script-file` option, ignore this option
      --run-bash-script <RUN_BASH_SCRIPT>
          run bash script
      --run-bash-script-file <RUN_BASH_SCRIPT_FILE>
          run bash script file when used together with the `--run-bash-script` option, ignore this option
  -p, --progress-bar
          enable progress bar
  -j, --build-workers <BUILD_WORKERS>
          build workers
  -J, --run-workers <RUN_WORKERS>
          run workers
      --in-place-template
          in place template
      --cache
          use cached workspaces, which means we only check the existence of the workspaces, and do not re-init the workspaces. you should make sure the workspaces are correct and up-to-date
  -h, --help
          Print help (see more with '--help')
  -V, --version
          Print version

License

This project is licensed under the MIT License - see the LICENSE file for details.