Struct ComputerControl 

pub struct ComputerControl {

    pub x: Option<i32>,
    pub y: Option<i32>,
    pub action: Option<PointerAction>,
    pub button: Option<String>,
    pub dx: Option<i32>,
    pub dy: Option<i32>,
    pub app: Option<String>,
    pub pid: Option<u32>,
    pub surface: Option<SurfaceKind>,
    pub selector: Option<String>,
    pub nth: Option<usize>,
    pub anchor: Option<PointerAnchor>,
    pub count: Option<u32>,
    pub held: Option<Vec<String>>,
    pub x2: Option<i32>,
    pub y2: Option<i32>,
    pub to_selector: Option<String>,
    pub duration_ms: Option<u64>,
    pub key: Option<String>,
    pub text: Option<String>,
    pub wait: Option<u64>,
    pub then: Option<Box<ComputerControl>>,
}

Input for computer_control — one step of a desktop-control pipeline: EITHER a pointer action (click/scroll/move/drag — key/text absent) OR a keyboard action (key or text), at raw coordinates or on an accessibility-tree element.

A step may chain the NEXT step via then: the steps run sequentially in a single call, each only if the previous succeeded (shell && semantics) — the first failing step aborts the chain with the point of failure. Targeting is PER STEP: the canonical pattern is click an element (the real click moves OS keyboard focus — the one mechanism that works on every toolkit) and then type into the focused field. up needs no target.

Fields

x: Option<i32>

X coordinate in desktop pixels (coordinate form; also the drag START).

y: Option<i32>

Y coordinate in desktop pixels (coordinate form; also the drag START).

action: Option<PointerAction>

Pointer action (default click). Ignored — and rejected together with any other pointer field — when the step is a keyboard step (key/text).

button: Option<String>

Button for down/up (left default, right, middle).

dx: Option<i32>

Horizontal scroll delta (wheel steps) for scroll.

dy: Option<i32>

Vertical scroll delta (wheel steps) for scroll; positive scrolls down.

app: Option<String>

Targeting PER STEP: application name (exact match) scoping selector. Provide app or pid — exactly one, only together with selector.

pid: Option<u32>

Application process ID for the element form. Per step.

surface: Option<SurfaceKind>

Target an OS SHELL SURFACE instead of an application (mutually exclusive with app/pid; only together with selector): the menu bar, Dock/taskbar/panel, tray status items, the desktop, or a flyout open right now. Pointer steps act on the element matched under the surface’s root the same way they do under an app-scoped step.

selector: Option<String>

CSS-like selector of the element this step acts on, e.g. "button[name='OK']" — the point resolves from the element’s CURRENT bounds when the step runs, never stale. A click on an element moves OS keyboard focus to it, which is how the next step’s typing finds the right field. Per step.

nth: Option<usize>

1-based match index when selector matches multiple elements (default 1). Requires selector. Per step.

anchor: Option<PointerAnchor>

Where inside the element’s bounds the point lands (default center).

count: Option<u32>

Number of consecutive clicks for click actions (1 default, 2 = double-click, 3 = triple). Independent of action: click + count: 2 is a double-click.

held: Option<Vec<String>>

Modifier keys for THIS step: held during a click or drag, or held while tapping key (shift, ctrl, alt, meta) — e.g. click + held: ["ctrl"] = ctrl+click; key a + held: ["ctrl"] = select all. text rejects held.

x2: Option<i32>

End X for drag (with y2). Mutually exclusive with to_selector.

y2: Option<i32>

End Y for drag.

to_selector: Option<String>

End element selector for drag (CSS-like selector on the same app) — the element form for the drag’s end point. Mutually exclusive with x2/y2.

duration_ms: Option<u64>

Total duration of a drag’s movement, in milliseconds (default 150, min 50). Backends interpolate the pointer path across this time.

key: Option<String>

Key to tap, by name: single characters (a, 5, . — lowercase; for uppercase hold shift), or enter, escape, tab, space, backspace, delete, insert, up, down, left, right, home, end, pageup, pagedown, f1..f12. Typing goes into the element focused by an earlier step’s element click (or whatever holds focus).

text: Option<String>

Literal text to type into the focused element (any characters, including uppercase). Mutually exclusive with key.

wait: Option<u64>

Milliseconds to WAIT as a STANDALONE step (no other fields): gives the app time to open a dialog or create an ephemeral field before the next step acts — e.g. click the menu item that opens a rename box, then {"wait": 600}, then type into it. Capped at 10 s.

then: Option<Box<ComputerControl>>

Optional NEXT step of the pipeline, executed only if this step succeeded (e.g. click text_field → then text "olá" → then key enter). Same shape as this step, recursively.

Trait Implementations

impl Clone for ComputerControl

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ComputerControl

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ComputerControl

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for ComputerControl

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl JsonSchema for ComputerControl

fn schema_name() -> Cow<'static, str>

The name of the generated JSON Schema.

fn schema_id() -> Cow<'static, str>

Returns a string that uniquely identifies the schema produced by this type.

fn json_schema(generator: &mut SchemaGenerator) -> Schema

Generates a JSON Schema for this type.

fn inline_schema() -> bool

Whether JSON Schemas generated for this type should be included directly in parent schemas, rather than being re-used where possible using the $ref keyword.