Module computer 

Desktop computer-control tools: observe native application UIs and interact with them through the OS accessibility tree (xa11y), screen captures, and synthetic pointer/keyboard input.

The layout separates COMPONENTS from PIPELINES:

• Components own one input KIND — validation and dispatch of a single step, reusable by any tool without carrying pipelines along: mouse (click/scroll/drag at coordinates or elements), keyboard (keys/text/wait + the shared input simulator) and touch (semantic accessibility actions on elements).
• Pipelines own one TOOL — chain walking (then with && semantics), step classification and the wire schema, composing the components: [control] (pointer + keyboard at raw coordinates or elements) and [act] (semantic + keyboard on accessibility-tree elements).
• Observation tools are standalone: [apps], [snapshot], [screenshot].

The two pipelines share the same schema skeleton on purpose (then → then → wait): the model learns the shape once and only the step kinds differ per tool.

All xa11y calls are blocking, so every operation runs on tokio’s blocking pool and returns an owned result.

Re-exports

pub use act::act;

pub use apps::AppsResult;

pub use apps::apps;

pub use control::control;

pub use screenshot::screenshot;

pub use snapshot::snapshot;

pub use types::ActAction;

pub use types::ActOutput;

pub use types::AppInfo;

pub use types::AppsOutput;

pub use types::AppsTarget;

pub use types::ComputerAct;

pub use types::ComputerApps;

pub use types::ComputerControl;

pub use types::ComputerScreenshot;

pub use types::ComputerSnapshot;

pub use types::ComputerWait;

pub use types::ControlOutput;

pub use types::ElementStates;

pub use types::FocusedElement;

pub use types::FocusedOutput;

pub use types::PointerAction;

pub use types::PointerAnchor;

pub use types::ScreenshotOutput;

pub use types::SnapshotFormat;

pub use types::SnapshotOutput;

pub use types::StateNode;

pub use types::ToggleState;

pub use types::WaitObservation;

pub use types::WaitOutput;

pub use types::WaitState;

pub use wait::wait;

Modules

act

computer_act — semantic actions AND keyboard as one pipeline on the accessibility tree (no screen coordinates involved).

apps

computer_apps — list running desktop applications, or report the foreground application and its keyboard-focused element.

control

computer_control — pointer AND keyboard as one pipeline tool.

errors

Error-rendering component — turns xa11y’s classified platform errors into messages that tell the model (and the user reading the trace) what to do next.

keyboard

Shared keyboard engine for the pipeline computer tools.

mouse

Pointer (mouse) component — validation and dispatch of pointer steps.

screenshot

computer_screenshot — capture the screen (or a region / element) and deliver the PNG inline as base64 through the connector’s multimodal channel.

snapshot

computer_snapshot — capture an application’s accessibility tree.

surface

Shell-surface component — resolution of an OS shell surface (menu bar, Dock/taskbar/panel, tray status items, desktop, flyouts) as a targeting root.

touch

Semantic (accessibility-action) component — validation and dispatch of element-action steps.

types

wait

computer_wait — block until an element reaches a state, or time out with a diagnosis.

Structs

Computer

Shared-state wrapper for computer-control tool operations.