droidrun-core

High-level Android device automation framework for Rust. Provides the DeviceDriver trait, DroidRun Portal integration, and a UI state processing pipeline.

Built on top of droidrun-adb. Part of the droidrun-rs workspace.

Features

• DeviceDriver trait — async interface for all device operations (tap, swipe, text, screenshot, UI tree)
• AndroidDriver — primary implementation using ADB + DroidRun Portal
• RecordingDriver — proxy wrapper that logs all actions as JSON
• Portal integration — dual-mode communication (TCP + ContentProvider fallback)
• UI state pipeline — accessibility tree filtering → formatting → structured UIState
• Portal management — automatic APK install, accessibility service setup, keyboard IME

Quick Start

[dependencies]
droidrun-core = "0.1"
tokio = { version = "1", features = ["full"] }

Basic Driver Usage

use droidrun_core::{AndroidDriver, DeviceDriver};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut driver = AndroidDriver::new(None, true); // TCP mode
    driver.connect().await?;

    // Screenshot
    let png = driver.screenshot(true).await?;
    std::fs::write("screen.png", &png)?;

    // Tap, type, navigate
    driver.tap(540, 1200).await?;
    driver.input_text("hello from rust!", false).await?;
    driver.press_key(4).await?; // Back

    // Get UI tree
    let state = driver.get_ui_tree().await?;
    println!("{}", serde_json::to_string_pretty(&state)?);

    Ok(())
}

UI State Pipeline

use droidrun_core::{AndroidDriver, DeviceDriver};
use droidrun_core::{AndroidStateProvider, ConciseFilter, IndexedFormatter};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut driver = AndroidDriver::new(None, true);
    driver.connect().await?;

    let provider = AndroidStateProvider::new(
        ConciseFilter,       // removes off-screen & tiny elements
        IndexedFormatter,    // assigns sequential indices
        false,               // use absolute coordinates
    );

    let state = provider.get_state(&driver).await?;

    println!("Screen: {}x{}", state.screen.width, state.screen.height);
    println!("Elements: {}", state.elements.len());
    println!("\n{}", state.formatted_text);

    // Find element by index
    if let Some(elem) = state.get_element(1) {
        println!("Element 1: {} '{}'", elem.class_name, elem.text);
    }

    Ok(())
}

Recording Driver

use droidrun_core::{AndroidDriver, RecordingDriver, DeviceDriver};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut inner = AndroidDriver::new(None, true);
    inner.connect().await?;

    let mut recorder = RecordingDriver::new(inner);

    recorder.tap(540, 1200).await?;
    recorder.input_text("hello", false).await?;
    recorder.press_key(4).await?;

    // Get recorded actions as JSON
    let actions = recorder.actions();
    println!("{}", serde_json::to_string_pretty(&actions)?);

    Ok(())
}

Architecture

┌──────────────────┐
│   Your App       │
└────────┬─────────┘
         │
┌────────▼─────────┐
│  DeviceDriver    │  async trait (tap, swipe, text, screenshot, UI tree)
│  ├ AndroidDriver │  ADB + Portal
│  └ RecordingDriver  Proxy with action logging
└────────┬─────────┘
         │
┌────────▼─────────┐
│  Portal Client   │  Dual-mode: TCP (fast) → ContentProvider (fallback)
│  Portal Manager  │  APK install, a11y service, keyboard IME
└────────┬─────────┘
         │
┌────────▼─────────┐
│  UI Pipeline     │  TreeFilter → TreeFormatter → UIState
│  ├ ConciseFilter │  Remove off-screen & tiny elements
│  └ IndexedFormatter  Assign indices, format text
└──────────────────┘

Portal Communication

DroidRun Portal is an Android APK that provides accessibility tree access, keyboard input, and screenshots.

Auto-fallback: if TCP fails, ContentProvider is used automatically.

UI State Output Format

index. ClassName: resourceId; checkedState, text - bounds(x1,y1,x2,y2)

Key Types

Examples

cargo run -p droidrun-core --example driver_basics
cargo run -p droidrun-core --example state_provider
cargo run -p droidrun-core --example recording
cargo run -p droidrun-core --example element_search
cargo run -p droidrun-core --example portal_setup
cargo run -p droidrun-core --example app_automation

Requirements

• Rust 1.85.0+
• ADB server running
• Android device/emulator with DroidRun Portal installed
• Portal accessibility service enabled

License

MIT