Skip to main content

Crate mobench_sdk

Crate mobench_sdk 

Source
Expand description

§mobench-sdk

Crates.io Documentation MIT License

A mobile benchmarking SDK for Rust that enables running performance benchmarks on real Android and iOS devices via BrowserStack App Automate.

§Overview

mobench-sdk provides a simple, declarative API for defining benchmarks that can run on mobile devices. It handles the complexity of cross-compilation, FFI bindings, and mobile app packaging automatically.

§Quick Setup Checklist

Before using mobench-sdk, ensure your project is configured correctly:

§Required Cargo.toml entries

[dependencies]
mobench-sdk = "0.1"
inventory = "0.3"  # Required for benchmark registration

[lib]
# Required for mobile FFI - produces .so (Android) and .a (iOS)
crate-type = ["cdylib", "staticlib", "lib"]

§When UniFFI is needed

If you’re creating custom FFI types for your benchmarks (custom errors, specs, etc.), you’ll also need UniFFI:

[dependencies]
uniffi = { version = "0.28", features = ["cli"] }
thiserror = "1.0"  # For custom error types
serde = { version = "1.0", features = ["derive"] }  # For serialization

[build-dependencies]
uniffi = { version = "0.28", features = ["build"] }

For most use cases, the SDK’s built-in types are sufficient and UniFFI setup is handled automatically by cargo mobench build.

§Troubleshooting

If benchmarks aren’t being discovered:

  1. Ensure functions are annotated with #[benchmark]
  2. Ensure functions are pub (public visibility)
  3. Ensure functions take no parameters and return ()
  4. Use the debug_benchmarks! macro to print registered benchmarks

For complete integration instructions, see BENCH_SDK_INTEGRATION.md

§Quick Start

§1. Add Dependencies

[dependencies]
mobench-sdk = "0.1"
inventory = "0.3"  # Required for benchmark registration

§2. Define Benchmarks

Use the #[benchmark] attribute to mark functions for benchmarking:

ⓘ
use mobench_sdk::benchmark;

#[benchmark]
fn my_expensive_operation() {
    let result = expensive_computation();
    std::hint::black_box(result);  // Prevent optimization
}

#[benchmark]
fn another_benchmark() {
    for i in 0..1000 {
        std::hint::black_box(i * i);
    }
}

§3. Build and Run

Use the mobench CLI to build and run benchmarks:

# Install the CLI
cargo install mobench

# Build for Android (outputs to target/mobench/)
cargo mobench build --target android

# Build for iOS
cargo mobench build --target ios

# Run on BrowserStack (use --release for smaller APK uploads)
cargo mobench run --target android --function my_expensive_operation \
    --iterations 100 --warmup 10 --devices "Google Pixel 7-13.0" --release

§Architecture

The SDK consists of several components:

ModuleDescription
timingCore timing infrastructure (always available)
registryRuntime discovery of #[benchmark] functions (requires full feature)
runnerBenchmark execution engine (requires full feature)
buildersAndroid and iOS build automation (requires full feature)
codegenMobile app template generation (requires full feature)
typesCommon types and error definitions

In repository CI, the SDK is typically exercised through the mobench CLI’s hybrid GitHub integration: compile-gated stateless controller workflows can start benchmark runs, while mobench-webhook owns history ingest and GitHub Check Runs when deployed.

§Crate Ecosystem

The mobench ecosystem consists of three published crates:

  • mobench-sdk (this crate) - Core SDK library with timing harness and build automation
  • mobench - CLI tool for building and running benchmarks
  • mobench-macros - #[benchmark] proc macro

Note: The mobench-runner crate has been consolidated into this crate as the timing module.

§Feature Flags

FeatureDefaultDescription
fullYesFull SDK with build automation, templates, and registry
runner-onlyNoMinimal timing-only mode for mobile binaries

For mobile binaries where binary size matters, use runner-only:

[dependencies]
mobench-sdk = { version = "0.1", default-features = false, features = ["runner-only"] }

§Programmatic Usage

You can also use the SDK programmatically:

§Using the Builder Pattern

ⓘ
use mobench_sdk::BenchmarkBuilder;

let report = BenchmarkBuilder::new("my_benchmark")
    .iterations(100)
    .warmup(10)
    .run()?;

println!("Mean: {} ns", report.samples.iter()
    .map(|s| s.duration_ns)
    .sum::<u64>() / report.samples.len() as u64);

§Using BenchSpec Directly

ⓘ
use mobench_sdk::{BenchSpec, run_benchmark};

let spec = BenchSpec {
    name: "my_benchmark".to_string(),
    iterations: 50,
    warmup: 5,
};

let report = run_benchmark(spec)?;
println!("Collected {} samples", report.samples.len());

§Discovering Benchmarks

ⓘ
use mobench_sdk::{discover_benchmarks, list_benchmark_names};

// Get all registered benchmark names
let names = list_benchmark_names();
for name in names {
    println!("Found benchmark: {}", name);
}

// Get full benchmark function info
let benchmarks = discover_benchmarks();
for bench in benchmarks {
    println!("Benchmark: {}", bench.name);
}

§Building Mobile Apps

The SDK includes builders for automating mobile app creation:

§Android Builder

ⓘ
use mobench_sdk::builders::AndroidBuilder;
use mobench_sdk::{BuildConfig, BuildProfile, Target};

let builder = AndroidBuilder::new(".", "my-bench-crate")
    .verbose(true)
    .output_dir("target/mobench");  // Default

let config = BuildConfig {
    target: Target::Android,
    profile: BuildProfile::Release,
    incremental: true,
};

let result = builder.build(&config)?;
println!("APK built at: {:?}", result.app_path);

§iOS Builder

ⓘ
use mobench_sdk::builders::{IosBuilder, SigningMethod};
use mobench_sdk::{BuildConfig, BuildProfile, Target};

let builder = IosBuilder::new(".", "my-bench-crate")
    .verbose(true);

let config = BuildConfig {
    target: Target::Ios,
    profile: BuildProfile::Release,
    incremental: true,
};

let result = builder.build(&config)?;
println!("xcframework built at: {:?}", result.app_path);

// Package IPA for distribution
let ipa_path = builder.package_ipa("BenchRunner", SigningMethod::AdHoc)?;

§Output Directory

By default, all mobile artifacts are written to target/mobench/:

target/mobench/
├── android/
│   ├── app/
│   │   ├── src/main/jniLibs/     # Native .so libraries
│   │   └── build/outputs/apk/    # Built APK
│   └── ...
└── ios/
    ├── sample_fns.xcframework/   # Built xcframework
    ├── BenchRunner/              # Xcode project
    └── BenchRunner.ipa           # Packaged IPA

This keeps generated files inside target/, following Rust conventions and preventing accidental commits of mobile project files.

§Platform Requirements

§Android

  • Android NDK (set ANDROID_NDK_HOME environment variable)
  • cargo-ndk (cargo install cargo-ndk)
  • Rust targets: rustup target add aarch64-linux-android armv7-linux-androideabi x86_64-linux-android

§iOS

  • Xcode with command line tools
  • uniffi-bindgen (cargo install uniffi-bindgen)
  • xcodegen (optional, brew install xcodegen)
  • Rust targets: rustup target add aarch64-apple-ios aarch64-apple-ios-sim x86_64-apple-ios

§Best Practices

§Use black_box to Prevent Optimization

Always wrap benchmark results with std::hint::black_box to prevent the compiler from optimizing away the computation:

ⓘ
#[benchmark]
fn correct_benchmark() {
    let result = expensive_computation();
    std::hint::black_box(result);  // Result is "used"
}

§Avoid Side Effects

Benchmarks should be deterministic and avoid I/O operations:

ⓘ
// Good: Pure computation
#[benchmark]
fn good_benchmark() {
    let data = vec![1, 2, 3, 4, 5];
    let sum: i32 = data.iter().sum();
    std::hint::black_box(sum);
}

// Avoid: File I/O adds noise
#[benchmark]
fn noisy_benchmark() {
    let data = std::fs::read_to_string("data.txt").unwrap();  // Don't do this
    std::hint::black_box(data);
}

§Choose Appropriate Iteration Counts

  • Warmup: 5-10 iterations to warm CPU caches and JIT
  • Iterations: 50-100 for stable statistics
  • Mobile devices may have more variance than desktop

§License

MIT License - see repository for details.

Re-exports§

pub use registry::BenchFunction;full
pub use registry::discover_benchmarks;full
pub use registry::find_benchmark;full
pub use registry::list_benchmark_names;full
pub use runner::BenchmarkBuilder;full
pub use runner::run_benchmark;full
pub use types::BenchError;
pub use types::BenchSample;
pub use types::BenchSpec;
pub use types::RunnerReport;
pub use types::BuildConfig;full
pub use types::BuildProfile;full
pub use types::BuildResult;full
pub use types::InitConfig;full
pub use types::Target;full
pub use timing::BenchSummary;
pub use timing::TimingError;
pub use timing::run_closure;
pub use inventory;full

Modules§

buildersfull
Build automation for mobile platforms.
codegenfull
Code generation and template management
ffi
Unified FFI module for UniFFI integration.
registryfull
Benchmark function registry
runnerfull
Benchmark execution runtime
timing
Lightweight benchmarking harness for mobile platforms.
types
Core types for mobench-sdk.
uniffi_types
UniFFI integration helpers for generating mobile bindings.

Macros§

debug_benchmarksfull
Generates a debug function that prints all discovered benchmarks.

Constants§

VERSION
Library version, matching Cargo.toml.

Functions§

black_box
Re-export of std::hint::black_box for preventing compiler optimizations.

Attribute Macros§

benchmarkfull
Marks a function as a benchmark for mobile execution.