1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
//! Agent-friendly input and screenshot control for Bevy apps.
//!
//! `bevy-agent-feedback-plugin` opens a local JSON-lines TCP socket that lets an
//! external Pi/Codex agent press keys, drive desktop-style mouse input, submit
//! text/file-drop events, wait for rendered frames, query primary-window
//! metadata, and capture the primary window. The plugin writes a protocol file
//! with the actual socket address so agents can discover a running app without
//! hard-coded ports.
//!
//! # Quick start
//!
//! ```no_run
//! use bevy::prelude::*;
//! use bevy_agent_feedback_plugin::{AgentFeedbackConfig, AgentFeedbackPlugin};
//! use std::{net::SocketAddr, path::PathBuf};
//!
//! App::new()
//! .add_plugins(DefaultPlugins)
//! .add_plugins(AgentFeedbackPlugin::new(AgentFeedbackConfig {
//! bind_addr: SocketAddr::from(([127, 0, 0, 1], 0)),
//! protocol_file: PathBuf::from("target/agent-feedback/agent-feedback.json"),
//! capture_dir: PathBuf::from("target/agent-feedback/captures"),
//! ..Default::default()
//! }))
//! .run();
//! ```
//!
//! After startup, read the configured protocol file. It contains protocol v2
//! session metadata, `socket_addr`, heartbeat path, capture directory, supported
//! commands, and example JSON requests. Send one newline-terminated JSON object
//! per request.
//!
//! # Protocol
//!
//! Requests have an `id` field and a `command` field. Valid responses echo the
//! `id`, set `ok`, and include either `result` or `error`; malformed requests
//! may return `id: null`.
//!
//! Supported commands include primitives (`key_down`, `mouse_down`,
//! `cursor_move`, `mouse_scroll`, `wait`, `capture`) and safe compound actions
//! (`click`, `drag`, `scroll`, `key_tap`, `key_hold`, `release_all_inputs`,
//! `shutdown`). Key commands target physical `KeyCode` input; apps should read
//! `ButtonInput<KeyCode>` or `KeyboardInput.key_code`. Cursor coordinates are
//! logical pixels in the primary window, with origin at the top-left.
//! Window/capture responses include frame, game time, window metadata, mouse
//! position, and currently agent-held inputs.
//!
//! # Scheduling
//!
//! Agent commands are drained in `PreUpdate` before Bevy's input systems. Normal
//! `Update` systems can read injected `ButtonInput`, accumulated mouse input,
//! window/input messages, and screenshot results.
//!
//! # Limits
//!
//! Queues, wait durations, request line length, captures, and
//! command response time are bounded by [`AgentFeedbackConfig`]. Keep the default
//! localhost bind address unless the control socket is protected from untrusted
//! clients.
//!
//! # Examples
//!
//! See the `examples/` directory for a minimal instrumented app and a
//! self-driving demo that uses the same protocol a Pi/Codex agent would use.
use *;
pub use AgentFeedbackConfig;
use AgentFeedbackControlPlugin;
pub use AgentFeedbackDiagnosticsPlugin;
use AgentFeedbackRuntimePlugin;
/// Bevy plugin that exposes local agent input and screenshot control.
///
/// Add this after the plugins that create input, window, and render resources.
/// `DefaultPlugins` is the usual choice for rendered examples.