mobench

Mobile benchmarking CLI for Rust - Run benchmarks on real Android and iOS devices.

The mobench CLI is the easiest way to benchmark your Rust code on mobile devices. It handles everything from project setup to building mobile apps to running tests on real devices via BrowserStack.

Installation

cargo binstall mobench

Or build from source:

cargo install mobench

Use as a Cargo subcommand:

cargo install mobench
cargo mobench --help

Quick Start

1. Initialize Your Project

# Create mobile benchmarking setup for Android
cargo mobench init --target android

# Or for iOS
cargo mobench init --target ios

# Or for both platforms
cargo mobench init --target both

This creates:

bench-mobile/ - FFI wrapper crate with UniFFI bindings
android/ or ios/ - Platform-specific app projects (generated to output directory)
bench-config.toml - Run configuration file
mobench.toml - Project configuration file (when using init)
benches/example.rs - Example benchmarks (with --examples)

2. Write Benchmarks

// benches/my_benchmarks.rs
use mobench_sdk::benchmark;

#[benchmark]
fn fibonacci_30() {
    let result = fibonacci(30);
    std::hint::black_box(result);
}

fn fibonacci(n: u32) -> u64 {
    match n {
        0 => 0,
        1 => 1,
        _ => fibonacci(n - 1) + fibonacci(n - 2),
    }
}

3. Build for Mobile

# Build Android APK
cargo mobench build --target android

# Build iOS app
cargo mobench build --target ios

4. Run Benchmarks

Local device workflow (builds artifacts and writes the run spec; launch the app manually):

cargo mobench run --target android --function fibonacci_30 --iterations 50

On real devices via BrowserStack:

export BROWSERSTACK_USERNAME=your_username
export BROWSERSTACK_ACCESS_KEY=your_access_key

cargo mobench run \
  --target android \
  --function fibonacci_30 \
  --devices "Google Pixel 7-13.0" \
  --iterations 100 \
  --warmup 10 \
  --release

Note: Always use the --release flag for BrowserStack runs. Debug builds are significantly larger (~544MB vs ~133MB for release) and may cause upload timeouts.

Commands

`init` - Initialize Project

Create mobile benchmarking infrastructure:

cargo mobench init [OPTIONS]

Options:

--target <android|ios|both> - Target platform (default: android)
--output <FILE> - Config file path (default: bench-config.toml)

Example:

cargo mobench init --target both --output my-bench.toml

`build` - Build Mobile Apps

Cross-compile and package for mobile platforms:

cargo mobench build --target <android|ios> [OPTIONS]

Options:

--target <android|ios> - Platform to build for (required)
--release - Build in release mode (default: debug)
--output-dir <DIR> - Output directory for mobile artifacts (default: target/mobench/)
--crate-path <PATH> - Path to the benchmark crate (default: auto-detect)
--dry-run - Print what would be done without making changes
--verbose / -v - Print verbose output including all commands

Examples:

# Build Android APK in release mode
cargo mobench build --target android --release

# Build iOS xcframework
cargo mobench build --target ios

# Preview build without making changes
cargo mobench build --target android --dry-run

# Build with verbose output
cargo mobench build --target ios --verbose

# Build to custom output directory
cargo mobench build --target android --output-dir ./my-output

Outputs:

Android: target/mobench/android/app/build/outputs/apk/debug/app-debug.apk
iOS: target/mobench/ios/sample_fns.xcframework

`run` - Run Benchmarks

Execute benchmarks on devices:

cargo mobench run --target <android|ios> --function <NAME> [OPTIONS]

Options:

--target <android|ios> - Platform (required)
--function <NAME> - Benchmark function name (required)
--iterations <N> - Number of iterations (default: 100)
--warmup <N> - Warmup iterations (default: 10)
--devices <LIST> - Comma-separated device list for BrowserStack
--device-matrix <FILE> - Load devices from a matrix YAML file (overrides device_matrix from --config when both are provided)
--device-tags <TAG> - Filter device matrix by tag (repeatable / comma-separated)
--local-only - Skip mobile builds (no device run)
--config <FILE> - Load run spec from config file
--ios-app <FILE> - iOS .ipa or zipped .app for BrowserStack
--ios-test-suite <FILE> - iOS XCUITest runner (.zip or .ipa)
--output <FILE> - Save results to JSON file (default: target/mobench/results.json)
--summary-csv - Write CSV summary alongside JSON/Markdown
--fetch - Fetch BrowserStack results after completion
--ci - CI mode (step summary + regression exit codes)
--baseline <path|url|artifact:<path>> - Compare against baseline summary (non-zero on regressions); if baseline resolves to the output file path, mobench snapshots the previous file first
--regression-threshold-pct <N> - Regression threshold percentage (default: 5.0)
--junit <FILE> - Write JUnit XML report

Outputs:

JSON summary (default: target/mobench/results.json)
Markdown summary (same base name, .md)
CSV summary (same base name, .csv, when --summary-csv is set)

Examples:

# Run locally (no BrowserStack devices specified)
cargo mobench run --target android --function fibonacci_30

# Run on BrowserStack devices (use --release for smaller APK)
cargo mobench run \
  --target android \
  --function sha256_hash \
  --devices "Google Pixel 7-13.0,Samsung Galaxy S23-13.0" \
  --iterations 50 \
  --release \
  --output results.json

# Run on iOS with auto-fetch (use --release for smaller artifacts)
cargo mobench run \
  --target ios \
  --function json_parse \
  --devices "iPhone 14-16,iPhone 15-17" \
  --release \
  --fetch

`ci run` - One-command CI Orchestration

Run build/package/run/fetch/report end-to-end with stable CI output files:

cargo mobench ci run --target <android|ios|both> --function <NAME> [OPTIONS]

Contract outputs (default directory: target/mobench/ci/):

summary.json
summary.md
results.csv

summary.json includes a ci section with metadata fields:

requested_by
pr_number
request_command
mobench_ref
mobench_version
trigger_source
dispatch_id

Contract references:

docs/CONTRACT_CI_V1.md
docs/schemas/summary-v1.schema.json
docs/schemas/ci-contract-v1.schema.json

When ci run is wrapped by the reusable GitHub workflow used in the 0.1.16 webhook flow, the canonical mobench-history-v1 bundle also stores per-platform workflow_inputs metadata such as base_ref and regression_threshold_pct for exact reruns and server-side comparisons.

Example:

cargo mobench ci run \
  --target android \
  --function sample_fns::fibonacci \
  --devices "Google Pixel 7-13.0" \
  --release \
  --fetch

# Combined android + ios contract output
cargo mobench ci run \
  --target both \
  --function sample_fns::fibonacci \
  --local-only

When --baseline is omitted for ci run, mobench automatically uses the previous successful summary snapshot in the target output directory when present.

`config validate` - Validate Run Config Contract

Validate bench-config.toml and referenced matrix/settings with contract-aligned issue categories:

cargo mobench config validate --config bench-config.toml
cargo mobench config validate --config bench-config.toml --format json

`devices resolve` - Deterministic Matrix Resolution

Resolve matrix devices for a platform/profile without custom scripts:

cargo mobench devices resolve \
  --platform android \
  --profile default \
  --device-matrix device-matrix.yaml

cargo mobench devices resolve \
  --platform ios \
  --config bench-config.toml \
  --format json

`fixture` - Fixture Lifecycle Commands

Manage reproducible fixture setup for CI:

# Create starter fixture files
cargo mobench fixture init

# Build fixture artifacts
cargo mobench fixture build --target both --release

# Verify fixture config + matrix resolution
cargo mobench fixture verify --config bench-config.toml

# Generate deterministic cache key
cargo mobench fixture cache-key --config bench-config.toml --format json

`package-ipa` - Package iOS IPA

Create a signed IPA for BrowserStack:

cargo mobench package-ipa [OPTIONS]

Options:

--scheme <NAME> - Xcode scheme (default: BenchRunner)
--method <adhoc|development> - Signing method (default: adhoc)

Example:

cargo mobench package-ipa --method adhoc

Output: target/mobench/ios/BenchRunner.ipa

`package-xcuitest` - Package XCUITest Runner

Create the XCUITest runner package required for BrowserStack iOS testing:

cargo mobench package-xcuitest [OPTIONS]

Options:

--scheme <NAME> - Xcode scheme for UI tests (default: BenchRunnerUITests)

Example:

cargo mobench package-xcuitest

Output: target/mobench/ios/BenchRunnerUITests.zip

This command builds the XCUITest target and packages it into the zip format that BrowserStack expects for iOS test automation.

`plan` - Generate Device Matrix

Create a template device matrix file:

cargo mobench plan [--output <FILE>]

Example:

cargo mobench plan --output devices.yaml

Output: device-matrix.yaml

devices:
  - name: Google Pixel 7-13.0
    os: android
    os_version: "13.0"
    tags: [default, pixel]
  - name: iPhone 14-16
    os: ios
    os_version: "16"
    tags: [default, iphone]

`list` - List Benchmarks

Show benchmarks discovered via #[benchmark]:

cargo mobench list

`fetch` - Fetch Results

Download BrowserStack build artifacts:

cargo mobench fetch --target <android|ios> --build-id <ID> [OPTIONS]

Options:

--target <android|ios> - Platform (required)
--build-id <ID> - BrowserStack build ID (required)
--output-dir <DIR> - Download directory (default: target/browserstack)

Example:

cargo mobench fetch \
  --target android \
  --build-id abc123def456 \
  --output-dir ./results

`compare` - Compare Summaries

Compare two JSON run summaries and emit a Markdown report:

cargo mobench compare \
  --baseline results-v1.json \
  --candidate results-v2.json \
  --output comparison.md

`report summarize` - Render CI Summary Markdown

Generate natural-language markdown from standardized output JSON:

cargo mobench report summarize --summary target/mobench/ci/summary.json
cargo mobench report summarize --summary target/mobench/ci/summary.json --output report.md

`report github` - Sticky PR Comment Payload/Publish

Create or update sticky PR comments from standardized outputs:

# Print comment body
cargo mobench report github --pr 123 --summary target/mobench/ci/summary.json

# Publish/update comment (requires GITHUB_TOKEN + GITHUB_REPOSITORY)
cargo mobench report github \
  --pr 123 \
  --summary target/mobench/ci/summary.json \
  --publish \
  --marker "<!-- mobench-report -->"

Configuration

Project Configuration (`mobench.toml`)

mobench automatically loads mobench.toml from the current directory or any parent directory:

[project]
# Name of the benchmark crate
crate = "bench-mobile"

# Rust library name (typically crate name with hyphens replaced by underscores)
library_name = "bench_mobile"

# Output directory for build artifacts (default: target/mobench/)
# output_dir = "target/mobench"

[android]
# Android package name
package = "com.example.bench"

# Minimum Android SDK version (default: 24)
min_sdk = 24

# Target Android SDK version (default: 34)
target_sdk = 34

[ios]
# iOS bundle identifier
bundle_id = "com.example.bench"

# iOS deployment target version (default: 15.0)
deployment_target = "15.0"

# Development team ID for code signing (optional)
# team_id = "YOUR_TEAM_ID"

[benchmarks]
# Default benchmark function to run
default_function = "my_crate::my_benchmark"

# Default number of benchmark iterations
default_iterations = 100

# Default number of warmup iterations
default_warmup = 10

CLI flags always override config file values when provided.

Run Config File Format (`bench-config.toml`)

For BrowserStack runs, you can also use a separate run configuration:

target = "android"
function = "sample_fns::fibonacci"
iterations = 100
warmup = 10
device_matrix = "device-matrix.yaml"
device_tags = ["default"] # optional; filter devices by tag

[browserstack]
app_automate_username = "${BROWSERSTACK_USERNAME}"
app_automate_access_key = "${BROWSERSTACK_ACCESS_KEY}"
project = "my-project-benchmarks"

[ios_xcuitest]
app = "target/mobench/ios/BenchRunner.ipa"
test_suite = "target/mobench/ios/BenchRunnerUITests.zip"

Device Matrix Format (`device-matrix.yaml`)

devices:
  - name: "Google Pixel 7-13.0"
    os: "android"
    os_version: "13.0"
    tags: ["default", "pixel"]
  - name: "iPhone 14-16"
    os: "ios"
    os_version: "16"
    tags: ["default", "iphone"]

Environment Variables

BrowserStack credentials can be provided via:

Environment variables (recommended):

export BROWSERSTACK_USERNAME=your_username
export BROWSERSTACK_ACCESS_KEY=your_access_key

.env.local file:

BROWSERSTACK_USERNAME=your_username
BROWSERSTACK_ACCESS_KEY=your_access_key

Config file with variable expansion:

[browserstack]
username = "${BROWSERSTACK_USERNAME}"
access_key = "${BROWSERSTACK_ACCESS_KEY}"

Requirements

For Android

Android NDK - Set ANDROID_NDK_HOME environment variable
cargo-ndk - Install with cargo install cargo-ndk
Android SDK - API level 24+ required
Gradle - For building APKs (bundled with Android project)

For iOS

macOS with Xcode installed
Xcode Command Line Tools - xcode-select --install

Rust iOS targets:

rustup target add aarch64-apple-ios
rustup target add aarch64-apple-ios-sim
rustup target add x86_64-apple-ios

XcodeGen - Install with brew install xcodegen

For BrowserStack

BrowserStack App Automate account - Sign up
Credentials - Username and access key from account settings

Examples

Benchmark Crypto Operations

# Initialize
cargo mobench init --target android

# Add benchmark
cat > benches/crypto.rs <<'EOF'
use mobench_sdk::benchmark;
use sha2::{Sha256, Digest};

#[benchmark]
fn sha256_1kb() {
    let data = vec![0u8; 1024];
    let hash = Sha256::digest(&data);
    std::hint::black_box(hash);
}
EOF

# Build
cargo mobench build --target android --release

# Run on multiple devices (use --release for BrowserStack)
cargo mobench run \
  --target android \
  --function sha256_1kb \
  --devices "Google Pixel 7-13.0,Samsung Galaxy S23-13.0,OnePlus 11-13.0" \
  --iterations 200 \
  --release \
  --output crypto-results.json

Compare iOS Performance

# Run same benchmark on different iOS versions (use --release for BrowserStack)
cargo mobench run \
  --target ios \
  --function json_parse \
  --devices "iPhone 13-15,iPhone 14-16,iPhone 15-17" \
  --iterations 100 \
  --release \
  --fetch \
  --output ios-comparison.json

CI Integration

Generate a ready-to-edit workflow + action wrapper:

cargo mobench ci init

This writes .github/workflows/mobile-bench.yml plus a local action in .github/actions/mobench/ that handles caching, Android setup, and artifact upload.

Example workflow excerpt:

- uses: ./.github/actions/mobench
  with:
    command: cargo mobench ci run
    run-args: |
      --target android
      --function my_benchmark
      --devices "Google Pixel 7-13.0"
      --iterations 50
      --release
      --fetch
    ci: false
  env:
    BROWSERSTACK_USERNAME: ${{ secrets.BROWSERSTACK_USERNAME }}
    BROWSERSTACK_ACCESS_KEY: ${{ secrets.BROWSERSTACK_ACCESS_KEY }}

The local action currently supports command values cargo mobench ci run and cargo mobench run.

For CI dashboards, add --junit path/to/results.junit.xml.

Typed Rust API

mobench also exposes a typed request/result surface for integrations:

use mobench::{DeviceSelection, MobileTarget, RunRequest, run_request};
use std::path::PathBuf;

let result = run_request(&RunRequest {
    target: MobileTarget::Android,
    function: "sample_fns::fibonacci".to_string(),
    iterations: 20,
    warmup: 5,
    device_selection: DeviceSelection {
        devices: vec!["Google Pixel 7-13.0".to_string()],
        device_matrix: None,
        device_tags: vec![],
    },
    config: None,
    baseline: None,
    regression_threshold_pct: 5.0,
    junit: None,
    local_only: true,
    release: false,
    ios_app: None,
    ios_test_suite: None,
    fetch: false,
    fetch_output_dir: PathBuf::from("target/browserstack"),
    fetch_poll_interval_secs: 5,
    fetch_timeout_secs: 300,
    progress: false,
    output_dir: PathBuf::from("target/mobench/ci"),
})?;
println!("summary: {}", result.report.summary_json.display());

Workflow

┌─────────────────────┐
│ 1. cargo mobench    │
│    init             │
└──────────┬──────────┘
           │
           ↓
┌─────────────────────┐
│ 2. Write benchmarks │
│    with #[benchmark]│
└──────────┬──────────┘
           │
           ↓
┌─────────────────────┐
│ 3. cargo mobench    │
│    build            │
└──────────┬──────────┘
           │
           ↓
┌─────────────────────┐
│ 4. cargo mobench    │
│    run              │
└──────────┬──────────┘
           │
      ┌────┴────┐
      ↓         ↓
┌──────────┐ ┌──────────────┐
│  Local   │ │ BrowserStack │
│ Emulator │ │ Real Devices │
└──────────┘ └──────────────┘

Troubleshooting

Android NDK not found

export ANDROID_NDK_HOME=/path/to/ndk

Or install via Android Studio: Tools → SDK Manager → SDK Tools → NDK

iOS code signing issues

For BrowserStack testing, use ad-hoc signing:

cargo mobench package-ipa --method adhoc

BrowserStack authentication failed

Verify credentials:

echo $BROWSERSTACK_USERNAME
echo $BROWSERSTACK_ACCESS_KEY

Or check .env.local file exists and contains valid credentials.

Benchmark function not found

Ensure:

Function has #[benchmark] attribute
Function is compiled into the mobile binary
Function name matches exactly (case-sensitive)

Part of mobench

This CLI is part of the mobench ecosystem:

mobench - This crate (CLI tool)
mobench-sdk - Core SDK with timing harness, build automation, and codegen
mobench-macros - #[benchmark] proc macro

License

Licensed under the MIT License. See LICENSE.md for details.

mobench 0.1.15-patch-1

mobench

Installation

Quick Start

1. Initialize Your Project

2. Write Benchmarks

3. Build for Mobile

4. Run Benchmarks

Commands

init - Initialize Project

build - Build Mobile Apps

run - Run Benchmarks

ci run - One-command CI Orchestration

config validate - Validate Run Config Contract

devices resolve - Deterministic Matrix Resolution

fixture - Fixture Lifecycle Commands

package-ipa - Package iOS IPA

package-xcuitest - Package XCUITest Runner

plan - Generate Device Matrix

list - List Benchmarks

fetch - Fetch Results

compare - Compare Summaries

report summarize - Render CI Summary Markdown

report github - Sticky PR Comment Payload/Publish

Configuration

Project Configuration (mobench.toml)

Run Config File Format (bench-config.toml)

Device Matrix Format (device-matrix.yaml)

Environment Variables

Requirements

For Android

For iOS

For BrowserStack

Examples

Benchmark Crypto Operations

Compare iOS Performance

CI Integration

Typed Rust API

Workflow

Troubleshooting

Android NDK not found

iOS code signing issues

BrowserStack authentication failed

Benchmark function not found

Part of mobench

See Also

License

`init` - Initialize Project

`build` - Build Mobile Apps

`run` - Run Benchmarks

`ci run` - One-command CI Orchestration

`config validate` - Validate Run Config Contract

`devices resolve` - Deterministic Matrix Resolution

`fixture` - Fixture Lifecycle Commands

`package-ipa` - Package iOS IPA

`package-xcuitest` - Package XCUITest Runner

`plan` - Generate Device Matrix

`list` - List Benchmarks

`fetch` - Fetch Results

`compare` - Compare Summaries

`report summarize` - Render CI Summary Markdown

`report github` - Sticky PR Comment Payload/Publish

Project Configuration (`mobench.toml`)

Run Config File Format (`bench-config.toml`)

Device Matrix Format (`device-matrix.yaml`)