Expand description
§gpui-mcp
Semantic inspection and Model Context Protocol (MCP) automation for GPUI applications.
The package contains both sides of the inspection seam:
- A small app-side library for publishing semantic state and receiving bounded UI actions.
- A
gpui-mcpstdio server that exposes that state to MCP clients.
It intentionally does not depend on GPUI itself. Your application is the adapter: it publishes framework-neutral nodes and executes requested actions on GPUI’s foreground executor.
§Application integration
Start the inspection endpoint only in explicit development or test builds.
Publish exactly one node with the application role and logical bounds; resize
and screenshot operations use that node as the semantic root.
use gpui_mcp::{
ActionRequest, InspectionRegistry, InspectionServer, SemanticBounds, SemanticNode,
SemanticTree,
};
fn start_inspection() -> Result<(), String> {
let registry = InspectionRegistry::default();
let (action_tx, action_rx) = async_channel::bounded::<ActionRequest>(16);
let server = InspectionServer::start(0, registry.clone(), action_tx)?;
registry.publish(SemanticTree {
revision: 1,
nodes: vec![SemanticNode {
id: "my-app.root".to_owned(),
role: "application".to_owned(),
label: "My GPUI app".to_owned(),
value: None,
bounds: Some(SemanticBounds {
x: 0.0,
y: 0.0,
width: 1024.0,
height: 640.0,
}),
focused: true,
actions: vec!["focus".to_owned()],
}],
});
println!("inspection address: {}", server.address());
println!("inspection token: {}", server.token());
// A real GPUI adapter drains `action_rx` and executes each request on the
// foreground executor before replying through `ActionRequest::response`.
drop(action_rx);
Ok(())
}The server binds only to IPv4 loopback, creates a fresh authentication token for every run, limits request sizes and action timeouts, and uses a bounded action channel. Treat its address and token as ephemeral credentials.
§MCP client configuration
Once installed with cargo install gpui-mcp, configure an MCP client to launch:
{
"mcpServers": {
"gpui": {
"command": "gpui-mcp",
"args": []
}
}
}Call attach with the address and token from the running application. The
server then provides semantic tree/query/wait operations, click and focus,
keyboard and text input, scrolling and dragging, resize settlement, and
logical-coordinate screenshots.
Prefer stable semantic IDs and exact selectors over screen coordinates. Verify semantic state after every action; screenshots are diagnostic evidence, not the functional assertion.
§License
Licensed under either Apache License 2.0 or the MIT license, at your option. GPUI semantic inspection state and authenticated loopback transport.
The listener is only constructed by opt-in inspection builds. It never executes application behavior itself: actions cross a bounded channel and are handled by the GPUI foreground executor.
Structs§
- Action
Request - An action that must be executed on the UI thread.
- Authenticated
Request - Inspection
Registry - Latest-value semantic registry. Reads never wait for GPUI.
- Inspection
Response - Inspection
Server - Running loopback inspection endpoint.
- Normalized
Modifiers - Semantic
Bounds - One normalized rectangle in logical pixels.
- Semantic
Node - One framework-neutral node exposed to inspection clients.
- Semantic
Point - Semantic
Selector - Exact, framework-neutral semantic node selector.
- Semantic
Tree - Immutable semantic snapshot published by the UI.
Enums§
- Inspection
Action - Inspection
Action Result - Inspection
Request - Semantic
Selector Error - Invalid semantic selector expression.
Functions§
- request
- Send one request to a running inspection endpoint.