terminator-rs 0.6.8

A Playwright-style SDK for automating desktop GUI applications
docs.rs failed to build terminator-rs-0.6.8
Please check the build logs for more information.
See Builds for ideas on how to fix a failed build, or Metadata for how to configure docs.rs builds.
If you believe this is docs.rs' fault, open an issue.

terminator πŸ€–

https://github.com/user-attachments/assets/00329105-8875-48cb-8970-a62a85a9ebd0

AI-powered desktop automation through natural language. Build agents that control any application using MCP (Model Context Protocol) + LLMs. Cross-platform SDK with Python, TypeScript, and Rust support.

⚑ TL;DR β€” Hello World Example

Skip the boilerplate. This is the fastest way to feel the magic.

πŸ€– Natural Language Control with MCP Client (Recommended)

Control your computer using natural language through Claude or other LLMs:

# Just talk to it:
πŸ’¬ You: Open Notepad and write a haiku about robots
πŸ€– Claude: I'll help you open Notepad and write a haiku...
   [Opening Notepad...]
   [Typing haiku...]
   βœ… Done! I've written a haiku about robots in Notepad.

πŸ’¬ You: Take a screenshot and tell me what apps are running
πŸ€– Claude: I'll capture your screen and analyze the running applications...
   [Taking screenshot...]
   βœ… I can see Chrome, VS Code, and Spotify are currently running.

Installation:

# Option 1: Download pre-built MCP agent (Windows)
# Download from: https://github.com/mediar-ai/terminator/releases/latest
# Extract terminator-mcp-windows-x86_64.zip and add to PATH

# Option 2: Build from source (requires Rust)
cargo build --release --bin terminator-mcp-agent

# Install Python dependencies
pip install mcp anthropic python-dotenv

# Set your API key and run
export ANTHROPIC_API_KEY='your-key-here'
python examples/python_mcp_client.py

See the full MCP client example β†’

🐍 Direct Python SDK

For programmatic control without AI:

import terminator

# Control applications programmatically
desktop = terminator.Desktop()
desktop.open_application('calc')
desktop.locator('name:Seven').click()
desktop.locator('name:Plus').click()  
desktop.locator('name:Three').click()
desktop.locator('name:Equals').click()
# Result: 10 appears in calculator

Installation:

pip install terminator.py

🟦 TypeScript / Node.js SDK

const { Desktop } = require('terminator.js');

// Async/await for modern control flow
const desktop = new Desktop();
await desktop.openApplication('notepad');
await desktop.locator('name:Edit').typeText('Hello from TypeScript!');
await desktop.pressKey('{Ctrl}s'); // Save

Installation:

npm install terminator.js
# or: bun add terminator.js

🧠 What is Terminator?

Terminator is an AI-first Playwright-style SDK for automating operating systems.

  • πŸͺŸ Built for Windows, with partial support on Linux and macOS
  • πŸ€– Learns deterministically from screen recordings of real workflows
  • 🧠 Designed for AI agentsβ€”not humans
  • ⚑ Uses OS-level accessibility APIs, with OCR/Vision as fallback
  • 🧩 Supports TypeScript, Python, MCP, and Rust
  • πŸ“ˆ Scans the UI in ~80msβ€”up to 10,000x faster and cheaper than a human

Terminator runs "headless" by default. It doesn't require a visible screen, relying instead on accessibility layers (like UI Automation on Windows) to interact with apps.

🎯 Building with MCP (Model Context Protocol)

The MCP integration is the recommended way to build AI-powered applications with Terminator. It provides a standardized interface between LLMs (like Claude) and desktop automation tools.

System Requirements

  • Windows: Pre-built binaries available (recommended)
  • macOS/Linux: Build from source with Rust
  • Python 3.8+ for the client
  • API Key from Anthropic, OpenAI, or other LLM provider

Why MCP?

  • Natural Language Interface: Control your desktop using plain English
  • AI-Powered Decision Making: The LLM decides which tools to use and when
  • Complex Workflows: Handle multi-step operations with a single command
  • Error Recovery: AI can adapt when things don't go as expected
  • Tool Chaining: Automatically sequences multiple operations
  • (Soon) Near determinism and production ready: We build the tools to make the MCP client near deterministic and production ready through generated workflows of human screen recordings

Quick Start with MCP

  1. Get the MCP agent:

    Option A: Download pre-built binary (Windows)

    • Download terminator-mcp-windows-x86_64.zip from latest releases
    • Extract and add to your PATH

    Option B: Build from source (requires Rust)

    cargo build --release --bin terminator-mcp-agent
    
  2. Run the Python MCP client:

    # Clone the repo if you haven't
    git clone https://github.com/mediar-ai/terminator
    cd terminator
    
    # Install dependencies
    pip install mcp anthropic python-dotenv
    
    # Set your Anthropic API key
    export ANTHROPIC_API_KEY='your-key-here'
    
    # Run the client
    python examples/python_mcp_client.py
    
  3. Try these examples:

    • "Open Chrome and search for 'best pizza near me'"
    • "Create a new text file on the desktop and write meeting notes"
    • "Take a screenshot and tell me what applications are running"
    • "Open Calculator and compute 15% tip on $47.50"

How It Works

The MCP client connects Claude (or other LLMs) to Terminator's automation capabilities:

You β†’ "Open Notepad and write a poem" β†’ Claude β†’ MCP Tools β†’ Desktop Automation
                                            ↓
                                    Decides to use:
                                    1. open_application
                                    2. wait_for_element
                                    3. type_into_element

Available MCP Tools

The MCP agent exposes 40+ tools for desktop automation, including:

  • Application Control: open_application, close_element, get_applications
  • UI Interaction: click_element, type_into_element, scroll_element
  • Navigation: wait_for_element, validate_element, get_window_tree
  • Data Capture: capture_screen, capture_element_screenshot, get_clipboard
  • Advanced: mouse_drag, press_key, select_option, set_range_value

Full MCP tool documentation β†’

Integrating with AI Applications

The MCP client example shows how to build conversational AI applications. You can:

  1. Build Custom Agents: Create specialized automation agents for specific workflows
  2. Integrate with Existing Apps: Add desktop automation to your AI applications
  3. Use Any LLM: While the example uses Claude, you can adapt it for GPT, Gemini, or open-source models
  4. Deploy at Scale: Run multiple MCP agents for different users or tasks

See the Python MCP client source for a complete implementation you can customize.

Direct SDK Usage (Alternative)

While MCP is recommended for AI-powered automation, you can also use the SDKs directly for programmatic control:

  • Python SDK: For long running tasks
  • TypeScript SDK: For web applications and Node.js services
  • Rust SDK: For high-performance system integration

These provide fine-grained control but require you to specify each action explicitly.

Feature Support

While Terminator aims for full cross-platform support, current capabilities vary by OS. Windows is the primary development target and has the most complete feature set.

Feature Windows macOS Linux Notes
Core Automation
Element Locators βœ… 🟑 🟑 Find elements by name, role, window, etc.
UI Actions (click, type) βœ… 🟑 🟑 Core interactions with UI elements.
Application Management βœ… 🟑 🟑 Launch, list, and manage applications.
Window Management βœ… 🟑 🟑 Get active window, list windows.
Advanced Features
Workflow Recording βœ… ❌ ❌ Record human workflows for deterministic automation.
Monitor Management βœ… 🟑 🟑 Multi-display support.
Screen & Element Capture βœ… βœ… 🟑 Take screenshots of displays or elements.
Language Bindings
Python (terminator.py) βœ… βœ… βœ… pip install terminator.py
TypeScript (terminator.js) βœ… βœ… βœ… npm i terminator.js
MCP (terminator-mcp-agent) βœ… βœ… βœ… npx -y terminator-mcp-agent --add-to-app [app]
Rust (terminator-rs) βœ… βœ… βœ… cargo add terminator-rs

Legend:

  • βœ…: Supported - The feature is stable and well-tested.
  • 🟑: Partial / Experimental - The feature is in development and may have limitations.
  • ❌: Not Supported - The feature is not yet available on this platform.

Documentation

For detailed information on features, installation, usage, and the API, please visit the Official Documentation.

Here's a section you can add under your README.md to document tools for inspecting accessibility elements across Windows, macOS, and Linux β€” tailored to Terminator users trying to find correct selectors:


πŸ•΅οΈ How to Inspect Accessibility Elements (like name:Seven)

To create reliable selectors (e.g. name:Seven, role:Button, window:Calculator), you need to inspect the Accessibility Tree of your OS. Here's how to explore UI elements on each platform:

πŸͺŸ Windows

  • Tool: Accessibility Insights for Windows
  • Alt: Inspect.exe (comes with Windows SDK)
  • Usage: Open the app you want to inspect β†’ launch Accessibility Insights β†’ hover or use keyboard navigation to explore the UI tree (Name, Role, ControlType, AutomationId).

These tools show you the Name, Role, ControlType, and other metadata used in Terminator selectors.


🍎 macOS

  • Tool: Accessibility Inspector
  • Usage: Comes with Xcode β†’ Open Xcode > Open Developer Tool > Accessibility Inspector β†’ Use the target icon to explore UI elements on screen.

🐧 Linux

  • Tool: Accerciser

  • Install:

    sudo apt install accerciser
    
  • Usage: Launch Accerciser β†’ Select the window/app β†’ Browse the accessible widget tree.


πŸ’‘ Tip

Once you identify the structure of your UI:

# Sample pattern
desktop.locator('window:Calculator')
       .locator('role:Button')
       .locator('name:Seven')

You can build and debug selector paths incrementally using .locator() chaining.

Explore Further

contributing

contributions are welcome! please feel free to submit issues and pull requests. many parts are experimental, and help is appreciated. join our discord to discuss.

businesses

if you want desktop automation at scale for your business, let's talk