🏋️‍♀️ bevy_rl

🏗️ Build 🤔 Reinforcement Learning 🏋🏿‍♂️ Gym environments with 🕊 Bevy engine to train 👾 AI agents that 💡 learn from 📺 screen pixels.

Compatibility

📝Features

• Set of APIs to implement OpenAI Gym interface
• REST API to control an agent
• Rendering to RAM membuffer

📋 Changelog

• 0.8.4
  • Added object representation of observation space
• 0.9.1
  • Bevy v.0.9 support
  • Minor changes in Deref ergonomics
• 0.9.3
  • Fixed a bug when AIGymState could not be initialized outside of the crate
• 0.9.4
  • Option to use crate without camera rendering to buffer

👩‍💻 Usage

1. Define App States

#[derive(Debug, Clone, Eq, PartialEq, Hash)]
enum AppState {
    InGame,  // where all the game logic is executed
    Control, // A paused state in which bevy_rl waits for agent actions
    Reset,   // A request to reset environment state
}

2. Define Action Space and Observation Space

A action space is a set of actions that an agent can take. An observation space is a set of observations that an agent can see. Action space can be discrete or continuous. Observations should be serializable to JSON with serde_json crate.

// Action space
#[derive(Default)]
pub struct Actions {
    ...
}

// Observation space
#[derive(Default, Serialize, Clone)]
pub struct State {
    ...
}

3. Enable AI Gym Plugin

Width and hight should exceed 256, otherwise wgpu will panic.

let gym_settings = AIGymSettings {
    width: 256,
    height: 256,
    num_agents: 16,
    no_graphics: false,
};

app
    .insert_resource(gym_settings.clone())
    .insert_resource(Arc::new(Mutex::new(AIGymState::<Actions,State>::new(gym_settings.clone()))))
    .add_plugin(AIGymPlugin::<Actions, State>::default())

4. Implement Environment Logic

DelayedControlTimer should pause environment execution to allow agents to take actions.

struct DelayedControlTimer(Timer);

Define systems that implement environment logic.

app.add_system_set(
    SystemSet::on_update(AppState::InGame)
        .with_system(turnbased_control_system_switch),
);

app.insert_resource(DelayedControlTimer(Timer::from_seconds(0.1, true))); // 10 Hz
app.add_system_set(
    SystemSet::on_update(AppState::Control)
        // Game Systems
        .with_system(turnbased_text_control_system) // System that parses user command
        .with_system(execute_reset_request),        // System that performs environment state reset
);

• turnbased_control_system_switch should pause game world and poll bevy_rl for agent actions.
• execute_reset_request handles environment reset request.
• turnbased_text_control_system parses agent actions and issues commands to agents in environment.

💻 AIGymState API

🌐 REST API

✍️ Examples

bevy_rl_shooter — example FPS project