Struct iai_callgrind::BinaryBenchmarkConfig

source ·

pub struct BinaryBenchmarkConfig(/* private fields */);

Expand description

The main configuration of a binary benchmark.

Currently it’s only possible to pass additional arguments to valgrind’s callgrind for all benchmarks. See BinaryBenchmarkConfig::raw_callgrind_args for more details.

Examples

use iai_callgrind::{BinaryBenchmarkConfig, main};

main!(
    config = BinaryBenchmarkConfig::default().raw_callgrind_args(["toggle-collect=something"]);
    binary_benchmark_groups = some_group
);

Implementations§

source §

impl BinaryBenchmarkConfig

source

pub fn fixtures<T>(&mut self, value: T) -> &mut Self
where T: Into<InternalFixtures>,

Copy Fixtures into the sandbox (if enabled)

See also Fixtures for details about fixtures and BinaryBenchmarkConfig::sandbox for details about the sandbox.

pub fn sandbox(&mut self, value: bool) -> &mut Self

Configure benchmarks to run in a sandbox (Default: true)

Per default, all binary benchmarks and the before, after, setup and teardown functions are executed in a temporary directory. This temporary directory will be created and changed into before the before function is run and removed after the after function has finished. BinaryBenchmarkConfig::fixtures let’s you copy your fixtures into that directory. If you want to access other directories within the benchmarked package’s directory, you need to specify absolute paths or set the sandbox argument to false.

Another reason for using a temporary directory as workspace is, that the length of the path where a benchmark is executed may have an influence on the benchmark results. For example, running the benchmark in your repository /home/me/my/repository and someone else’s repository located under /home/someone/else/repository may produce different results only because the length of the first path is shorter. To run benchmarks as deterministic as possible across different systems, the length of the path should be the same wherever the benchmark is executed. This crate ensures this property by using the tempfile crate which creates the temporary directory in /tmp with a random name of fixed length like /tmp/.tmp12345678. This ensures that the length of the directory will be the same on all unix hosts where the benchmarks are run.

Examples

use iai_callgrind::{main, BinaryBenchmarkConfig};

main!(
    config = BinaryBenchmarkConfig::default().sandbox(false);
    binary_benchmark_groups = my_group
);

source

pub fn raw_callgrind_args<I, T>(&mut self, args: T) -> &mut Self
where I: AsRef<str>, T: IntoIterator<Item = I>,

Pass arguments to valgrind’s callgrind

It’s not needed to pass the arguments with flags. Instead of --collect-atstart=no simply write collect-atstart=no.

See also Callgrind Command-line Options for a full overview of possible arguments.

Examples

use iai_callgrind::BinaryBenchmarkConfig;

let config = BinaryBenchmarkConfig::default()
    .raw_callgrind_args(["collect-atstart=no", "toggle-collect=some::path"]);

source

pub fn env<K, V>(&mut self, key: K, value: V) -> &mut Self
where K: Into<OsString>, V: Into<OsString>,

Add an environment variable

These environment variables are available independently of the setting of BinaryBenchmarkConfig::env_clear.

Examples

An example for a custom environment variable “FOO=BAR”:

use iai_callgrind::{main, BinaryBenchmarkConfig};

main!(
    config = BinaryBenchmarkConfig::default().env("FOO", "BAR");
    binary_benchmark_groups = my_group
);

source

pub fn envs<K, V, T>(&mut self, envs: T) -> &mut Self
where K: Into<OsString>, V: Into<OsString>, T: IntoIterator<Item = (K, V)>,

Add multiple environment variable available in this Run

Examples

use iai_callgrind::{main, BinaryBenchmarkConfig};

main!(
    config = BinaryBenchmarkConfig::default().envs([("FOO", "BAR"),("BAR", "BAZ")]);
    binary_benchmark_groups = my_group
);

source

pub fn pass_through_env<K>(&mut self, key: K) -> &mut Self
where K: Into<OsString>,

Specify a pass-through environment variable

Usually, the environment variables before running a binary benchmark are cleared but specifying pass-through variables makes this environment variable available to the benchmark as it actually appeared in the root environment.

Pass-through environment variables are ignored if they don’t exist in the root environment.

Examples

Here, we chose to pass-through the original value of the HOME variable:

use iai_callgrind::{main, BinaryBenchmarkConfig};

main!(
    config = BinaryBenchmarkConfig::default().pass_through_env("HOME");
    binary_benchmark_groups = my_group
);

source

pub fn pass_through_envs<K, T>(&mut self, envs: T) -> &mut Self
where K: Into<OsString>, T: IntoIterator<Item = K>,

Specify multiple pass-through environment variables

Examples

use iai_callgrind::{main, BinaryBenchmarkConfig};

main!(
    config = BinaryBenchmarkConfig::default().pass_through_envs(["HOME", "USER"]);
    binary_benchmark_groups = my_group
);

source

pub fn env_clear(&mut self, value: bool) -> &mut Self

If false, don’t clear the environment variables before running the benchmark (Default: true)

Examples

use iai_callgrind::{main, BinaryBenchmarkConfig};

main!(
    config = BinaryBenchmarkConfig::default().env_clear(false);
    binary_benchmark_groups = my_group
);

source

pub fn current_dir<T>(&mut self, value: T) -> &mut Self
where T: Into<PathBuf>,

Set the directory of the benchmarked binary (Default: Unchanged)

Unchanged means, in the case of running with the sandbox enabled, the root of the sandbox. In the case of running without sandboxing enabled, this’ll be the root of the package directory of the benchmark. If running the benchmark within the sandbox, and the path is relative then this new directory must be contained in the sandbox.

Examples

use iai_callgrind::{main, BinaryBenchmarkConfig};

main!(
    config = BinaryBenchmarkConfig::default().current_dir("/tmp");
    binary_benchmark_groups = my_group
);

and the following will change the current directory to fixtures assuming it is contained in the root of the sandbox

use iai_callgrind::{main, BinaryBenchmarkConfig};

main!(
    config = BinaryBenchmarkConfig::default().current_dir("fixtures");
    binary_benchmark_groups = my_group
);

source

pub fn entry_point<T>(&mut self, value: T) -> &mut Self
where T: Into<String>,

Set the start and entry point for event counting of the binary benchmark run

Per default, the counting of events starts right at the start of the binary and stops when it finished execution. This’ll include some os specific code which makes the executable actually runnable. To focus on what is actually happening inside the benchmarked binary, it may desirable to start the counting for example when entering the main function (but can be any function) and stop counting when leaving the main function of the executable. The following example will show how to do that.

Examples

The entry_point could look like my_exe::main for a binary with the name my-exe (Note that hyphens are replaced with an underscore).

use iai_callgrind::{main, BinaryBenchmarkConfig};

main!(
    config = BinaryBenchmarkConfig::default().entry_point("my_exe::main");
    binary_benchmark_groups = my_group
);

About: How to find the right entry point

If unsure about the entry point, it is best to start without setting the entry point and inspect the callgrind output file of the benchmark of interest. These are usually located under target/iai. The file format is completely documented here. To focus on the lines of interest for the entry point, these lines start with fn=.

The example above would include a line which would look like fn=my_exe::main with information about the events below it and maybe some information about the exact location of this function above it.

Now, you can set the entry point to what is following the fn= entry. To stick to the example, this would be my_exe::main. Running the benchmark again should now show the event counts of everything happening after entering the main function and before leaving it. If the counts are 0 (and the main function is not empty), something went wrong and you have to search the output file again for typos or similar.

source

pub fn exit_with<T>(&mut self, value: T) -> &mut Self
where T: Into<InternalExitWith>,

Set the expected exit status ExitWith of a benchmarked binary

Per default, the benchmarked binary is expected to succeed which is the equivalent of ExitWith::Success. But, if a benchmark is expected to fail, setting this option is required.

Examples

If the benchmark is expected to fail with a specific exit code, for example 100:

use iai_callgrind::{main, BinaryBenchmarkConfig, ExitWith};

main!(
    config = BinaryBenchmarkConfig::default().exit_with(ExitWith::Code(100));
    binary_benchmark_groups = my_group
);

If a benchmark is expected to fail, but the exit code doesn’t matter:

use iai_callgrind::{main, BinaryBenchmarkConfig, ExitWith};

main!(
    config = BinaryBenchmarkConfig::default().exit_with(ExitWith::Failure);
    binary_benchmark_groups = my_group
);

source

pub fn flamegraph<T>(&mut self, config: T) -> &mut Self
where T: Into<InternalFlamegraphConfig>,

Option to produce flamegraphs from callgrind output using the crate::FlamegraphConfig

Examples

use iai_callgrind::{main, BinaryBenchmarkConfig, FlamegraphConfig };

main!(
    config = BinaryBenchmarkConfig::default().flamegraph(FlamegraphConfig::default());
    binary_benchmark_groups = my_group
);

source

pub fn regression<T>(&mut self, config: T) -> &mut Self
where T: Into<InternalRegressionConfig>,

Enable performance regression checks with a crate::RegressionConfig

Examples

use iai_callgrind::{main, BinaryBenchmarkConfig, RegressionConfig};

main!(
    config = BinaryBenchmarkConfig::default().regression(RegressionConfig::default());
    binary_benchmark_groups = my_group
);

source

pub fn tool<T>(&mut self, tool: T) -> &mut Self
where T: Into<InternalTool>,

Add a configuration to run a valgrind crate::Tool in addition to callgrind

Examples

use iai_callgrind::{main, BinaryBenchmarkConfig, Tool, ValgrindTool};

main!(
    config = BinaryBenchmarkConfig::default()
        .tool(
            Tool::new(ValgrindTool::DHAT)
        );
    binary_benchmark_groups = my_group
);

source

pub fn tools<I, T>(&mut self, tools: T) -> &mut Self
where I: Into<InternalTool>, T: IntoIterator<Item = I>,

Add multiple configurations to run valgrind crate::Tools in addition to callgrind

Examples

use iai_callgrind::{main, BinaryBenchmarkConfig, Tool, ValgrindTool};

main!(
    config = BinaryBenchmarkConfig::default()
        .tools(
            [
                Tool::new(ValgrindTool::DHAT),
                Tool::new(ValgrindTool::Massif)
            ]
        );
    binary_benchmark_groups = my_group
);

source

pub fn tool_override<T>(&mut self, tool: T) -> &mut Self
where T: Into<InternalTool>,

Override previously defined configurations of valgrind crate::Tools

See also crate::LibraryBenchmarkConfig::tool_override for more details.

Example

The following will run DHAT and Massif (and the default callgrind) for all benchmarks in main! besides for foo which will just run Memcheck (and callgrind).

use iai_callgrind::{
    binary_benchmark_group, Run, BinaryBenchmarkConfig, main, Tool, ValgrindTool, Arg
};

binary_benchmark_group!(
    name = my_group;
    benchmark = |"my-exe", group: &mut BinaryBenchmarkGroup| {
        group.bench(
            Run::with_arg(Arg::new("foo", &["foo"]))
                .tool_override(Tool::new(ValgrindTool::Memcheck))
        );
    }
);

main!(
    config = BinaryBenchmarkConfig::default()
        .tools(
            [
                Tool::new(ValgrindTool::DHAT),
                Tool::new(ValgrindTool::Massif)
            ]
        );
    binary_benchmark_groups = my_group
);

source

pub fn tools_override<I, T>(&mut self, tools: T) -> &mut Self
where I: Into<InternalTool>, T: IntoIterator<Item = I>,

Override previously defined configurations of valgrind crate::Tools

See also crate::LibraryBenchmarkConfig::tool_override for more details.

Example

The following will run DHAT (and the default callgrind) for all benchmarks in main! besides for foo which will run Massif and Memcheck (and callgrind).

use iai_callgrind::{
    binary_benchmark_group, Run, BinaryBenchmarkConfig, main, Tool, ValgrindTool, Arg
};

binary_benchmark_group!(
    name = my_group;
    benchmark = |"my-exe", group: &mut BinaryBenchmarkGroup| {
        group.bench(
            Run::with_arg(Arg::new("foo", &["foo"]))
                .tools_override([
                    Tool::new(ValgrindTool::Massif),
                    Tool::new(ValgrindTool::Memcheck),
                ])
        );
    }
);

main!(
    config = BinaryBenchmarkConfig::default()
        .tool(
            Tool::new(ValgrindTool::DHAT),
        );
    binary_benchmark_groups = my_group
);