rs_peekaboo

Rust-native cross-platform computer-use CLI and library for macOS, Windows, and Linux.

rs_peekaboo focuses on capture, UI inspection, input, app/window/menu control, clipboard, permissions, scripts, and structured JSON output. It intentionally does not include model providers, hosted API keys, telemetry, or an assistant UI.

Install

From crates.io after release:

cargo install rs_peekaboo

From source:

git clone https://github.com/undivisible/rs_peekaboo
cd rs_peekaboo
cargo build --release
cargo install --path .

CLI

rs-peekaboo image --mode screen --path ~/Desktop/screen.png
rs-peekaboo image --app Safari --path ~/Desktop/safari.png
rs-peekaboo see --app Safari --json
rs-peekaboo click --coords 500,300
rs-peekaboo click --index 3 --snapshot snap-... --background
rs-peekaboo type "hello" --return
rs-peekaboo hotkey cmd,l
rs-peekaboo window list --json
rs-peekaboo app launch --app Safari
rs-peekaboo clipboard read --json
rs-peekaboo doctor --json
rs-peekaboo shell "fastfetch --logo none" --json
rs-peekaboo mcp   # MCP stdio server for agent hosts

Global flags:

Commands:

JSON Output

rs-peekaboo image --mode screen --json

{
  "ok": true,
  "data": {
    "path": "/var/folders/.../rs_peekaboo_x.png",
    "mode": "screen",
    "bytes": 123456,
    "mime_type": "image/png"
  }
}

rs-peekaboo see --app Safari --json

{
  "ok": true,
  "data": {
    "snapshot_id": "snap-...",
    "elements": [
      {
        "id": "window:Safari:Example",
        "role": "window",
        "label": "Example",
        "app": "Safari",
        "window": "Example",
        "index": 0,
        "bounds": {
          "x": 0,
          "y": 25,
          "width": 1200,
          "height": 800
        },
        "state": {
          "minimized": false
        }
      }
    ]
  }
}

Click by stable snapshot index:

rs-peekaboo click --index 0 --snapshot snap-... --background --json

Library

use rs_peekaboo::automation::Target;
use rs_peekaboo::{Bounds, Direction, ImageMode, Peekaboo, Point};

fn main() -> rs_peekaboo::Result<()> {
    let peekaboo = Peekaboo::new();

    let image = peekaboo.image(ImageMode::Screen, None, true)?;
    let region = peekaboo.image_region(
        Bounds {
            x: 0,
            y: 0,
            width: 800,
            height: 600,
        },
        None,
        true,
    )?;
    let elements = peekaboo.ui_elements(None)?;

    peekaboo.click(Target::Point(Point { x: 500, y: 300 }), "left", 1)?;
    peekaboo.type_text("hello", false, false, None, None)?;
    peekaboo.hotkey(&["cmd", "l"])?;
    peekaboo.scroll(Direction::Down, 3)?;
    peekaboo.window("focus", Some("Safari"), None, None)?;
    peekaboo.app("launch", Some("Safari"))?;
    peekaboo.menu("list", "Safari", None, None)?;
    peekaboo.clipboard_write("copied")?;

    println!("{} {} {}", image.bytes, region.bytes, elements.len());
    Ok(())
}

The v0.2.4 source shapes and method signatures are available through a namespaced compatibility facade. Migrating a v0.2.4 caller requires changing its imports; the current root API remains unchanged.

use rs_peekaboo::compat::{Peekaboo, Target};
use rs_peekaboo::{ImageMode, Point};

fn main() -> rs_peekaboo::Result<()> {
    let peekaboo = Peekaboo;
    let image = peekaboo.image(ImageMode::Screen, None, true)?;
    peekaboo.click(Target::Point(Point { x: 500, y: 300 }), "left", 1)?;
    println!("{}", image.bytes);
    Ok(())
}

Scripts

run accepts a JSON file with ordered steps:

{
  "steps": [
    {
      "command": "hotkey",
      "args": {
        "keys": "cmd,l"
      }
    },
    {
      "command": "type",
      "args": {
        "text": "https://example.com"
      }
    },
    {
      "command": "sleep",
      "args": {
        "duration_ms": 250
      }
    }
  ]
}

rs-peekaboo run ./script.json --json

Shell steps are also supported:

{
  "steps": [
    {
      "command": "shell",
      "args": {
        "command": "pwd",
        "cwd": "/tmp"
      }
    }
  ]
}

Permissions

Most actions require macOS Accessibility permission for the terminal or host app running rs-peekaboo. Screen capture requires Screen Recording permission. Clipboard commands require clipboard access in the current user session.

Probe status:

rs-peekaboo permissions status --json

Platform

Shell execution is cross-platform and uses the host shell.

Folk Around

rs_peekaboo is standalone. It is designed so MCP hosts such as folk-around can call the Rust computer-use engine directly instead of shelling out to AppleScript helpers.

License

ISC.

Acknowledgements

rs_peekaboo began as an independent Rust rewrite of Peekaboo. Its background driver and MCP compatibility surface also draws on public interface patterns from Cua Driver.