Struct ComputerUseDriver 

pub struct ComputerUseDriver { /* private fields */ }

Shared driver all seven computer-use tools hold an Arc<> to.

Implementations

impl ComputerUseDriver

pub fn new(backend: Backend) -> Self

pub fn backend(&self) -> Backend

pub fn ensure_alive(&self) -> Result<(), String>

Cheap mid-call liveness check. Tools call this first inside execute(); if the display went away after registration, they return a clean error instead of hanging on subprocess dispatch.

pub fn scale_coords( &self, x: i32, y: i32, screenshot_id: Option<u64>, ) -> Result<(i32, i32), String>

Translate model-space coords to screen-space using the metadata registered for screenshot_id (or the latest if None).

pub fn register_screenshot( &self, scale_factor: f64, offset_x: i32, offset_y: i32, kind: String, ) -> u64

Allocate a fresh registry id and record metadata.

pub async fn capture( &self, spec: ScreenshotSpec, token: &CancellationToken, ) -> Result<CaptureResult>

Capture and return the encoded result. Respects cancellation via token.cancelled() races in the subprocess wait.

pub async fn capture_focused_for_autoshot( &self, token: &CancellationToken, ) -> Option<(String, String)>

Convenience for click/type/key tools: capture the focused window and return (summary, base64_png) for inclusion in the tool’s auto-screenshot. Best-effort — on error returns None and the caller can fall back to a screenshot-less outcome.

pub async fn check_cursor_landed(&self, sx: i32, sy: i32) -> Option<String>

X11-only: verify the cursor actually landed where xdotool was told to move it. Returns Some(warning) if the cursor ended up more than CURSOR_LANDED_TOLERANCE_PX away (focus change, window moved, WM rejected the move). None if within tolerance or the probe itself failed (best-effort — never blocks the click).

impl ComputerUseDriver

pub async fn click( &self, sx: i32, sy: i32, button: &str, token: &CancellationToken, ) -> Result<()>

Click at the given SCREEN coordinates (already scaled by scale_coords). button is "left" | "middle" | "right".

pub async fn type_text( &self, text: &str, token: &CancellationToken, ) -> Result<()>

Type text at the current focus. Per-keystroke delay from TYPE_KEY_DELAY_MS — empirically needed for slow Electron / web targets that drop characters at lower rates.

pub async fn press_key( &self, key: &str, token: &CancellationToken, ) -> Result<()>

Press a key (or key combination like "ctrl+shift+t").

pub async fn scroll( &self, direction: &str, amount: i32, token: &CancellationToken, ) -> Result<()>

Scroll amount ticks in direction (“up” / “down”).

pub async fn mouse_move( &self, sx: i32, sy: i32, token: &CancellationToken, ) -> Result<()>

Move the mouse cursor to SCREEN coords (already scaled).

pub async fn list_windows( &self, _token: &CancellationToken, ) -> Result<Vec<String>>

List visible window titles. X11 only; Wayland has no portable enumeration primitive.