tauri-plugin-agent-control

Dev-only HTTP bridge for Tauri webviews — like Chrome DevTools Protocol, but for Tauri.

What it does

tauri-plugin-agent-control exposes your Tauri webview over a local HTTP API at http://localhost:9876. You can snapshot the DOM, click buttons, fill inputs, evaluate JavaScript, intercept network requests, take screenshots, and more — all via simple curl commands or any HTTP client. The server only runs in debug builds, so it's completely stripped from production.

Installation

1. Add the Rust dependency

From crates.io:

[dependencies]
tauri-plugin-agent-control = "0.1"

Or from git:

[dependencies]
tauri-plugin-agent-control = { git = "https://github.com/palanik1/tauri-plugin-agent-control" }

2. Register the plugin

In your src-tauri/src/lib.rs (where tauri::Builder is configured):

tauri::Builder::default()
    .plugin(tauri_plugin_agent_control::init())
    // ... your other plugins and handlers
    .run(tauri::generate_context!())
    .expect("error while running tauri application");

3. Add capability permissions

In your src-tauri/capabilities/default.json (create it if it doesn't exist):

{
  "identifier": "default",
  "windows": ["main"],
  "permissions": [
    "agent-control:default"
  ]
}

agent-control:default grants allow-agent-control-respond — the only Tauri command the plugin registers. It allows the JS shim to send eval results back to the Rust HTTP server.

4. Run in dev mode

cargo tauri dev

The HTTP bridge starts automatically on http://localhost:9876 in debug builds. Verify it's running:

curl -s http://localhost:9876/health
# {"ok":true}

Production builds: The server is completely stripped — it's gated behind #[cfg(debug_assertions)]. No code, no port, no surface area.

AI Agent Setup

Install the Amp/AI agent skill

Copy the SKILL.md from this package into your project's skill directory:

# Project-level (recommended)
mkdir -p .agents/skills/tauri-agent-control
cp node_modules/tauri-plugin-agent-control/SKILL.md .agents/skills/tauri-agent-control/SKILL.md

# Or user-level (available across all projects)
mkdir -p ~/.config/agents/skills/tauri-agent-control
cp node_modules/tauri-plugin-agent-control/SKILL.md ~/.config/agents/skills/tauri-agent-control/SKILL.md

Or install via npm and copy:

npm install tauri-plugin-agent-control
cp node_modules/tauri-plugin-agent-control/SKILL.md .agents/skills/tauri-agent-control/SKILL.md

Once installed, your AI agent (Amp, Claude Code, etc.) can interact with your running Tauri app via natural language:

• "Take a screenshot of the app"
• "Click the Submit button"
• "Fill in the email field with test@example.com"
• "Check if the sidebar is visible"

The agent uses the skill to translate these into the appropriate curl commands to the HTTP bridge.

Quick Start

1. Snapshot the page to see what's on screen:

curl http://localhost:9876/snapshot?format=compact

[page] My App — http://localhost:1420/
  @e1 [button] "Submit" role="button"
  @e2 [input type="text"] placeholder="Enter name" name="username"
  @e3 [a] "Home" href="/"

2. Interact with elements using their refs:

# Fill an input
curl -X POST http://localhost:9876/fill \
  -d '{"ref":"@e2","text":"hello world"}'

# Click a button
curl -X POST http://localhost:9876/click \
  -d '{"ref":"@e1"}'

3. Re-snapshot to see the updated state:

curl http://localhost:9876/snapshot?format=compact

4. Evaluate arbitrary JavaScript:

curl -X POST http://localhost:9876/eval \
  -d '{"code":"document.title"}'

API Reference

Snapshot

Interactions

Getters

State Checks

Wait

Eval

Console / Errors

Cookies

Storage

Navigation

Find (Semantic Locators)

All /find calls return {"ref":"@eN","tag":"...","text":"..."} and accept optional "action" (click, fill, type, hover, focus), "text" (for fill/type), and "exact":true for exact text matching.

Mouse / Keyboard

Highlight

Dialogs

Viewport

State

Screenshot

Recording

Download

Network

Geolocation

Offline

Headers

Trace

Health

Ref Lifecycle

Every call to /snapshot or /find assigns elements a ref like @e1, @e2, etc. These refs are invalidated whenever you take a new snapshot — the ref map is cleared and rebuilt from scratch. If the page navigates or the DOM changes significantly, take a fresh snapshot before interacting.

# 1. Snapshot → get refs
curl http://localhost:9876/snapshot?format=compact
# @e1 [button] "Save"

# 2. Use the ref
curl -X POST http://localhost:9876/click -d '{"ref":"@e1"}'

# 3. After DOM changes, snapshot again for fresh refs
curl http://localhost:9876/snapshot?format=compact
# @e1 [button] "Saved ✓"   ← refs are reassigned

Semantic Locators (Find)

The /find endpoint lets you locate elements without needing a snapshot first. It assigns a new ref to the found element and optionally performs an action in one call.

# Find a button by role and click it
curl -X POST http://localhost:9876/find \
  -d '{"by":"role","value":"button","name":"Submit","action":"click"}'

# Find an input by label and fill it
curl -X POST http://localhost:9876/find \
  -d '{"by":"label","value":"Email","action":"fill","text":"user@example.com"}'

# Find by test ID
curl -X POST http://localhost:9876/find \
  -d '{"by":"testid","value":"search-input","action":"type","text":"query"}'

# Find the 3rd list item (0-based index)
curl -X POST http://localhost:9876/find \
  -d '{"by":"nth","value":"li","index":2}'

Supported locator strategies: role, text, label, placeholder, testid, alt, title, first, last, nth.

Use "exact":true for exact text/name matching instead of substring matching.

Network Interception

Intercept, mock, and block network requests made by the webview:

# Start intercepting
curl -X POST http://localhost:9876/network/intercept

# Mock an API response
curl -X POST http://localhost:9876/network/route \
  -d '{"url":"/api/users","body":"{\"users\":[{\"name\":\"Alice\"}]}"}'

# Block requests to analytics
curl -X POST http://localhost:9876/network/route \
  -d '{"url":"analytics.example.com","abort":true}'

# View captured requests
curl http://localhost:9876/network/requests

# Reset (restore original fetch/XHR)
curl -X POST http://localhost:9876/network/reset

The interceptor patches window.fetch and XMLHttpRequest. Tauri IPC calls (ipc:// URLs) are automatically excluded to avoid breaking invoke().

Security

• Debug-only: The HTTP server is gated behind #[cfg(debug_assertions)] and is completely compiled out of release builds.
• Localhost-only: The server listens on localhost:9876. It is not accessible from other machines.
• Not for production: This plugin gives full control over the webview — eval, DOM manipulation, cookie/storage access. Never ship it in a release build.

Platform Notes

Architecture

The plugin is composed of three parts:

1. Rust HTTP server — A raw TCP server built with tokio::net::TcpListener that manually parses HTTP/1.1 requests. It runs as a spawned async task during plugin setup (debug builds only).

2. JS shim (guest-js/index.js) — Embedded at compile time via include_str!("../guest-js/index.js"). It's injected into the webview on every navigation using Tauri's on_navigation hook with a 200ms delay to let the new page settle. The shim exposes window.__DEVTOOLS__ with all DOM interaction methods.

3. Eval bridge — The communication channel between Rust and JS:

   • Rust generates a unique 16-hex-char request ID
   • Rust calls webview.eval() with wrapped JS that executes the expression
   • The JS result is sent back via window.__TAURI_INTERNALS__.invoke('plugin:agent-control|agent_control_respond', {requestId, data})
   • Rust receives the result through a tokio::sync::oneshot channel
   • A configurable timeout (default 5s) cleans up pending requests

┌──────────┐   HTTP    ┌────────────┐  eval()   ┌───────────┐
│  Client  │ ───────── │ Rust TCP   │ ────────── │  Webview  │
│ (curl)   │           │ Server     │            │  (JS)     │
└──────────┘           └────────────┘  invoke()  └───────────┘
                             ▲ ──────────────────────┘
                             │  agent_control_respond(id, data)
                             │  via oneshot channel

License

MIT