pub enum Action {
Show 54 variants
Screenshot {
output: Option<String>,
region: Option<Region>,
format: Option<ImageFormat>,
max_width: Option<u32>,
canvas: Option<String>,
base64: bool,
marks: bool,
},
Click {
x: Option<i32>,
y: Option<i32>,
button: MouseButton,
double: bool,
count: u8,
},
TripleClick {
x: Option<i32>,
y: Option<i32>,
button: MouseButton,
},
DoubleClick {
x: Option<i32>,
y: Option<i32>,
button: MouseButton,
},
Press {
count: u8,
button: MouseButton,
},
Drag {
path: Vec<Point>,
button: MouseButton,
},
MouseDown {
x: i32,
y: i32,
button: MouseButton,
},
MouseUp {
x: i32,
y: i32,
button: MouseButton,
},
HoldKey {
keys: String,
ms: u64,
},
ClipboardGet,
ClipboardSet {
text: Option<String>,
image: Option<String>,
},
Zoom {
output: Option<String>,
region: Region,
format: Option<ImageFormat>,
max_width: Option<u32>,
canvas: Option<String>,
base64: bool,
},
Focus,
SetValue {
value: String,
},
SelectText {
start: i64,
end: i64,
},
PerformAction {
name: String,
},
ScrollTo,
Move {
x: i32,
y: i32,
},
Type {
text: String,
},
Key {
keys: String,
},
Scroll {
x: Option<i32>,
y: Option<i32>,
dx: Option<i32>,
dy: Option<i32>,
direction: Option<ScrollDirection>,
amount: Option<i32>,
},
GetState,
Wait {
ms: Option<u64>,
seconds: Option<f64>,
},
Displays,
Isolation,
ListApps {
all: bool,
},
ListWindows,
Launch,
Activate,
Quit {
force: bool,
},
PolicyCheck,
Pause,
Resume,
Doctor {
prompt: bool,
open_settings: bool,
x11: bool,
},
Tree,
SessionEnd,
CursorPosition,
Batch {
stop_on_error: bool,
actions: Vec<Value>,
},
SessionPrune {
dry_run: bool,
ttl_secs: Option<u64>,
},
MenuList {
app: Option<String>,
depth: Option<u32>,
},
MenuClick {
app: Option<String>,
path: String,
},
AppleEvent {
app: Option<String>,
script: String,
language: Option<AppleEventLanguage>,
},
ShortcutsRun {
name: String,
},
DockList {},
DockClick {
title: String,
},
Spaces {},
BrowserTabs {},
BrowserActivate {
target_id: Option<String>,
},
BrowserNavigate {
url: String,
target_id: Option<String>,
},
BrowserClick {
selector: String,
target_id: Option<String>,
},
BrowserText {
selector: String,
target_id: Option<String>,
},
BrowserType {
text: String,
selector: Option<String>,
target_id: Option<String>,
},
BrowserEval {
script: String,
target_id: Option<String>,
},
Ocr {
force: bool,
},
}Expand description
One desktop action. Serialized with an action tag.
{"action":"click","x":10,"y":20}Variants§
Screenshot
Capture the screen or a region.
Fields
format: Option<ImageFormat>png (default) or jpeg.
Click
Move to a point and click, or press an element when element is set.
Fields
Button. Defaults to left.
TripleClick
Click three times.
Fields
Button. Defaults to left.
DoubleClick
Click twice. Equivalent to click with double: true.
Fields
Button. Defaults to left.
Press
Press an element. count is 1, 2, or 3.
Fields
Mouse button. Left uses AXPress. Right uses AXShowMenu when advertised.
Drag
Button down at the first point, dragged events through the rest, up at the last.
Fields
Mouse button. Defaults to left.
MouseDown
Button down. A later mouse_up, or the end of a one-shot command, releases it.
MouseUp
Button up.
HoldKey
Key down, sleep ms, key up. One combination. ms above 30000 is refused.
ClipboardGet
Read the clipboard. Text wins over an image. Empty is kind: empty.
ClipboardSet
Write the clipboard. Confirmed. The audit stores the character count, not the text.
Fields
Zoom
Screenshot of a region. A window target crops that region in window-local points.
Fields
format: Option<ImageFormat>png (default) or jpeg.
Focus
Set AXFocused on an element.
SetValue
Set an element’s value. Refused for secure fields.
SelectText
Set the selected character range. Refused for secure fields.
PerformAction
Perform an allowlisted accessibility action.
ScrollTo
Scroll an element into view.
Move
Move the pointer without clicking.
Type
Type text into the focused control.
Key
Press a key combination such as cmd+shift+4.
Scroll
Scroll the wheel. Positive dy is down. Positive dx is right.
Fields
direction: Option<ScrollDirection>Direction used with amount.
GetState
Screen, cursor, frontmost window, and a best-effort accessibility tree.
Wait
Sleep, then return.
Fields
Displays
List displays without capturing pixels.
Isolation
Report what this build can isolate and what it cannot.
ListApps
List running applications. all includes accessory and prohibited apps.
ListWindows
List on-screen windows. app comes from the shared request context.
Launch
Launch an application named by the shared app field.
Activate
Activate an application named by the shared app field.
Quit
Ask an application to quit. force requests a forced terminate.
PolicyCheck
Report whether policy would allow app. Does not perform an action.
Pause
Pause mutating actions until resume.
Resume
Clear the pause flag.
Doctor
Report permissions and the TCC identity.
The process exits 0 when the report is produced. permissions_ok in the
body is the permission aggregate. prompt asks macOS to show dialogs.
request is an alias of prompt.
Fields
Tree
Numbered accessibility tree for the key window.
SessionEnd
Restore Electron accessibility flags and stop observers.
CursorPosition
Read the pointer. The Anthropic adapter maps cursor_position here.
Batch
Run several actions that share one target and one cache revision sequence.
Fields
SessionPrune
Remove empty session directories and stale ones.
Fields
MenuList
Walk an application’s menu bar. Reads AXMenuBar -> AXMenuBarItem -> AXMenu.
--app selects the running app. The default depth is 4; depth caps it.
Matches are case-insensitive. Disabled items are still reported.
Fields
MenuClick
Click a menu item by ">"-separated titles.
--app selects the running app. --path is a list of titles such as
"File > Save As > Save". Matches are case-insensitive. Disabled items
are not pressed.
Fields
AppleEvent
Send an Apple event via osascript. macOS only.
--app is recorded and used by the deny policy; the script body decides
which application the event targets. --language is applescript (the
default) or javascript. The script body is never logged verbatim.
Fields
language: Option<AppleEventLanguage>applescript (default) or javascript.
ShortcutsRun
Run a Shortcuts workflow by name. macOS only.
The shortcuts binary is called with the workflow name as a single
argument. A missing binary is execution_failed.
DockList
List the Dock items. macOS only.
DockClick
Click a Dock item by title. macOS only.
Spaces
Group the on-screen windows by the screen that contains each window’s center.
Mission Control moves are not attempted; the report only describes the current layout. The skylight feature, when active, may move windows between spaces; otherwise the action reports the visible state only.
BrowserTabs
List the tabs the running Chrome exposes. Requires --features cdp.
Without the feature this returns unsupported.
BrowserActivate
Activate a tab by target_id. Requires --features cdp.
Navigate the named tab (or first page) to url. The URL is checked
against [browser].blocked_urls before the websocket is touched.
BrowserClick
Click an element matched by CSS selector. Requires --features cdp.
BrowserText
Read the innerText of the first element matched by CSS selector.
Requires --features cdp. The current page URL is checked against
[browser].blocked_urls first; a selector that matches nothing returns
element_not_found.
Fields
BrowserType
Type text into the element matched by selector (or the page body
when no selector is given). Requires --features cdp. Classified like
Action::Type for audit: the body is never stored, only its
character count. The current page URL is checked against
[browser].blocked_urls first.
Fields
BrowserEval
Evaluate script in the page context and return the JSON value.
Requires --features cdp. The script body is never logged verbatim;
only its length and a small hash appear in the audit row.
Fields
Ocr
Run OCR on the most recent capture for --app/window target. macOS
only. Linux returns unsupported; Windows is a stub until P12.