Module agent_api 

Agent API endpoints.

These endpoints allow AI agents to interact with the running application through structured HTTP requests, without needing native desktop control.

How it works

1. The browser app includes the rdesktop bridge script
2. The bridge script periodically sends DOM snapshots to the server
3. Agents query the server for DOM/state information
4. Agents send actions to the server, which forwards them to the browser

Design Philosophy

Instead of requiring agents to take screenshots and use vision models to understand the UI, the Agent API provides direct DOM access and structured state information. This is:

• Faster: No screenshot encoding/decoding overhead
• More reliable: Exact element selectors, not pixel coordinates
• More informative: Full DOM tree, computed styles, accessibility info
• Easier to test: Standard HTTP endpoints, can be scripted

Structs

ActionQuery

ActionResult

Result of an action execution.

ActionResultReport

AgentAction

An action that an agent can execute on the UI.

DomSnapshot

Response from a DOM query.

ElementInfo

Response from an element query.

ElementQuery

Query parameters for element selection.

RecordingCompleteRequest

RecordingErrorRequest

RecordingStartRequest

Optional request body for starting a recording. The server owns the recording identity; agents may safely send {} more than once.

RecordingStartedRequest

RecordingStopRequest

Optional session guard for stopping a recording.

ScreenshotQuery

GET /rdesktop/agent/screenshot

Enums

ActionType

Types of actions agents can execute.

Functions

execute_action

POST /rdesktop/agent/action

get_dom

GET /rdesktop/agent/dom

get_recording

GET /rdesktop/agent/recording

get_state

GET /rdesktop/agent/state

pending_actions

GET /rdesktop/agent/action/pending

poll_recording

GET /rdesktop/agent/recording/poll

query_elements

GET /rdesktop/agent/elements?selector=…

recording_chunk

POST /rdesktop/agent/recording/chunk

recording_complete

POST /rdesktop/agent/recording/complete

recording_error

POST /rdesktop/agent/recording/error

recording_file

GET /rdesktop/agent/recording/file

recording_started

POST /rdesktop/agent/recording/started

report_action_result

POST /rdesktop/agent/action/result

send_ipc

POST /rdesktop/agent/ipc

start_recording

POST /rdesktop/agent/recording/start

stop_recording

POST /rdesktop/agent/recording/stop

take_screenshot

Capture the latest complete native PNG frame.