gpui-mcp 0.1.0

Semantic inspection and MCP automation for GPUI applications
Documentation

gpui-mcp

Semantic inspection and Model Context Protocol (MCP) automation for GPUI applications.

The package contains both sides of the inspection seam:

  • A small app-side library for publishing semantic state and receiving bounded UI actions.
  • A gpui-mcp stdio server that exposes that state to MCP clients.

It intentionally does not depend on GPUI itself. Your application is the adapter: it publishes framework-neutral nodes and executes requested actions on GPUI's foreground executor.

Application integration

Start the inspection endpoint only in explicit development or test builds. Publish exactly one node with the application role and logical bounds; resize and screenshot operations use that node as the semantic root.

use gpui_mcp::{
    ActionRequest, InspectionRegistry, InspectionServer, SemanticBounds, SemanticNode,
    SemanticTree,
};

fn start_inspection() -> Result<(), String> {
    let registry = InspectionRegistry::default();
    let (action_tx, action_rx) = async_channel::bounded::<ActionRequest>(16);
    let server = InspectionServer::start(0, registry.clone(), action_tx)?;

    registry.publish(SemanticTree {
        revision: 1,
        nodes: vec![SemanticNode {
            id: "my-app.root".to_owned(),
            role: "application".to_owned(),
            label: "My GPUI app".to_owned(),
            value: None,
            bounds: Some(SemanticBounds {
                x: 0.0,
                y: 0.0,
                width: 1024.0,
                height: 640.0,
            }),
            focused: true,
            actions: vec!["focus".to_owned()],
        }],
    });

    println!("inspection address: {}", server.address());
    println!("inspection token: {}", server.token());

    // A real GPUI adapter drains `action_rx` and executes each request on the
    // foreground executor before replying through `ActionRequest::response`.
    drop(action_rx);
    Ok(())
}

The server binds only to IPv4 loopback, creates a fresh authentication token for every run, limits request sizes and action timeouts, and uses a bounded action channel. Treat its address and token as ephemeral credentials.

MCP client configuration

Once installed with cargo install gpui-mcp, configure an MCP client to launch:

{
  "mcpServers": {
    "gpui": {
      "command": "gpui-mcp",
      "args": []
    }
  }
}

Call attach with the address and token from the running application. The server then provides semantic tree/query/wait operations, click and focus, keyboard and text input, scrolling and dragging, resize settlement, and logical-coordinate screenshots.

Prefer stable semantic IDs and exact selectors over screen coordinates. Verify semantic state after every action; screenshots are diagnostic evidence, not the functional assertion.

License

Licensed under either Apache License 2.0 or the MIT license, at your option.