Module agent_api 

Agent API endpoints.

These endpoints allow AI agents to interact with the running application through structured HTTP requests, without needing native desktop control.

How it works

1. The browser app includes the rdesktop bridge script
2. The bridge script periodically sends DOM snapshots to the server
3. Agents query the server for DOM/state information
4. Agents send actions to the server, which forwards them to the browser

Design Philosophy

Instead of requiring agents to take screenshots and use vision models to understand the UI, the Agent API provides direct DOM access and structured state information. This is:

• Faster: No screenshot encoding/decoding overhead
• More reliable: Exact element selectors, not pixel coordinates
• More informative: Full DOM tree, computed styles, accessibility info
• Easier to test: Standard HTTP endpoints, can be scripted

Structs

ActionResult

Result of an action execution.

AgentAction

An action that an agent can execute on the UI.

DomSnapshot

Response from a DOM query.

ElementInfo

Response from an element query.

ElementQuery

Query parameters for element selection.

Enums

ActionType

Types of actions agents can execute.

Functions

execute_action

POST /rdesktop/agent/action

get_dom

GET /rdesktop/agent/dom

get_state

GET /rdesktop/agent/state

query_elements

GET /rdesktop/agent/elements?selector=…

send_ipc

POST /rdesktop/agent/ipc

take_screenshot

GET /rdesktop/agent/screenshot