1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
//! Computer-use subsystem — RsClaw's GUI agent core.
//!
//! Architecture (5 layers, top-down):
//!
//! Layer A. Third-party primitives (`enigo` + `xcap`):
//! Native cross-platform input synthesis and screen capture.
//! Replaces the previous shell-based path (cliclick / PowerShell
//! / sips / screencapture) with ~100x faster native APIs.
//!
//! Layer B. Operator trait (`operators/`):
//! Platform abstraction. Implementations:
//! - `NativeOperator` — desktop (Mac/Win/Linux) via enigo+xcap
//! - `BrowserOperator` — bridge to web_browser subsystem (CDP)
//! - (future) `AdbOperator` — Android
//! Each operator self-describes its capabilities via
//! `action_spaces()` so the system prompt is built dynamically.
//!
//! Layer C. Driver (`driver.rs`, `parser.rs`, `prompt.rs`):
//! Model-agnostic AI loop. VlmDriver works with any vision model
//! that follows the Thought/Action format (UI-TARS 1.0/1.5,
//! Doubao, GPT-4o, Claude vision, Qwen-VL, ...). Coordinate
//! parser is format-tolerant (4 formats: <|box_start|>, <point>,
//! (x,y), [x1,y1,x2,y2]).
//!
//! Layer D. Permission gate (`permission.rs`):
//! Pre-execution consent flow. Before any UI loop runs the
//! backend emits a PermissionRequest event; the desktop UI
//! surfaces a modal ("RsClaw is about to control WeChat, ~10
//! steps") and the user grants once / for the session / always
//! (per-app) / denies. Decisions persist in redb.
//!
//! Layer E. App rules (`app_rules.rs`, runtime data):
//! Plain markdown files in `tools/computer_use/app-rules/`.
//! Loaded at runtime, matched by keyword to the user's
//! instruction, and injected into the system prompt. Adding a
//! new app's automation knowledge does NOT require Rust changes.
pub use ;
pub use ;
pub use Operator;
pub use ;
/// Build the platform-default operator for the running OS. Browser /
/// ADB operators are constructed explicitly elsewhere (they need
/// driver-specific configuration).