Skip to main content

Crate bevy_agent_feedback_plugin

Crate bevy_agent_feedback_plugin 

Source
Expand description

Agent-friendly input, timing, diagnostics, and screenshot control for Bevy apps.

bevy-agent-feedback-plugin exposes a bounded local protocol for input, semantic game readiness, deterministic Bevy time, and completion-confirmed primary-window PNGs. Protocol readiness means only that the socket exists; animated games should wait on a registered predicate/target and then capture after one app update. Strict pixel stability is appropriate only for static content or a bounded static region.

§Quick start

Add feedback after DefaultPlugins (or after the plugins that provide Bevy time, window, render, and input resources):

use bevy::prelude::*;
use bevy_agent_feedback_plugin::{AgentFeedbackConfig, AgentFeedbackPlugin};
use std::{net::SocketAddr, path::PathBuf};

App::new()
    .add_plugins(DefaultPlugins)
    .add_plugins(AgentFeedbackPlugin::new(AgentFeedbackConfig {
        bind_addr: SocketAddr::from(([127, 0, 0, 1], 0)),
        protocol_file: PathBuf::from("target/agent-feedback/agent-feedback.json"),
        capture_dir: PathBuf::from("target/agent-feedback/captures"),
        deterministic_time: true,
        ..Default::default()
    }))
    .run();

The generated protocol file is authoritative for the 0.5 socket, heartbeat, commands, deterministic-time mode, and caps. The wire envelope remains {"id":...,"command":...} with responses containing ok and either result or error.

§Timing, capture, and diagnostics

Public clients call wait_frames for app-update progress; the compatibility wire command remains "wait". App updates are neither elapsed gameplay time nor compositor-presented frames. wait_seconds observes normal Bevy virtual time and is rejected while deterministic time is frozen. advance_time progresses deterministic Bevy virtual/fixed time in bounded nominal steps. Deterministic mode does not control direct std::time::Instant, OS/network clocks, unseeded RNG, or other external state.

capture_after_frames and wait_until_first_capture return metadata for screenshot readback: sequence/path/label, request/completion app-update frames, physical image dimensions, request/completion window state, and screenshot_captured completion. Readback and PNG persistence do not prove that the OS/window compositor presented the image.

With the diagnostics feature, AgentFeedbackDiagnosticsPlugin explicitly registers states, scalar resource fields, and marker components for predicates and semantic targets. Exact duplicate Name/accessibility/marker targets error rather than choosing the first. Input/target coordinates are logical window pixels; PNG masks and crop regions are physical image pixels.

§Scheduling and limits

Agent commands enter in PreUpdate before Bevy input consumers. Screenshot responses complete only after Bevy render readback. Queues, request lines, waits, actions, time advances, diagnostics scans, captures, and response time are bounded by AgentFeedbackConfig and advertised in the protocol file. Keep the localhost bind unless the socket is protected from untrusted clients.

§Canonical workflow

See skills/driving-bevy-games/SKILL.md for setup, readiness selection, semantic waits, input defaults, visual fallbacks, and artifact handling. See examples/agent_driven.rs for the self-driving Rust client.

Modules§

client
Rust client for the 0.5 agent feedback protocol.

Structs§

AgentFeedbackConfig
Runtime configuration for crate::AgentFeedbackPlugin.
AgentFeedbackPlugin
Bevy plugin that exposes local agent input and screenshot control.

Enums§

DiagnosticValue
Bounded scalar value exposed by registered diagnostic resource fields.