Struct PpoAgent 

pub struct PpoAgent<E, Enc, Act, B>
where
    E: Environment,
    B: AutodiffBackend,
{ /* private fields */ }

A PPO agent with discrete actions.

Implements clipped-surrogate PPO with GAE advantage estimation and an actor-critic network. Generic over environment, encoder, action mapper, and Burn backend – the same pattern as DqnAgent.

On-policy: experience is collected into a rollout buffer, used for n_epochs gradient passes, then discarded. The buffer size is n_steps * n_envs; an update fires automatically when it fills.

Parallel environments

Set PpoConfig::n_envs to match the number of environments feeding this agent (e.g. bevy-gym’s NUM_ENVS). All envs contribute to the same rollout buffer; the update fires after n_steps ticks.

Implementations

impl<E, Enc, Act, B> PpoAgent<E, Enc, Act, B>

where E: Environment, E::Observation: Clone + Send + Sync + 'static, E::Action: Clone + Send + Sync + 'static, Enc: ObservationEncoder<E::Observation, B> + ObservationEncoder<E::Observation, B::InnerBackend>, Act: DiscreteActionMapper<E::Action>, B: AutodiffBackend,

pub fn new( encoder: Enc, action_mapper: Act, config: PpoConfig, device: B::Device, seed: u64, ) -> Self

Trait Implementations

impl<E, Enc, Act, B> Checkpointable for PpoAgent<E, Enc, Act, B>

where E: Environment, E::Observation: Clone + Send + Sync + 'static, E::Action: Clone + Send + Sync + 'static, Enc: ObservationEncoder<E::Observation, B> + ObservationEncoder<E::Observation, B::InnerBackend>, Act: DiscreteActionMapper<E::Action>, B: AutodiffBackend,

fn save(&self, path: &Path) -> Result<()>

Save weights to path (without extension – implementations add their own).

fn load(self, path: &Path) -> Result<Self>

Load weights from path, consuming and returning self.

impl<E, Enc, Act, B> LearningAgent<E> for PpoAgent<E, Enc, Act, B>

where E: Environment, E::Observation: Clone + Send + Sync + 'static, E::Action: Clone + Send + Sync + 'static, Enc: ObservationEncoder<E::Observation, B> + ObservationEncoder<E::Observation, B::InnerBackend>, Act: DiscreteActionMapper<E::Action>, B: AutodiffBackend,

fn act(&mut self, obs: &E::Observation, mode: ActMode) -> E::Action

Select an action for obs according to mode.

fn observe(&mut self, experience: Experience<E::Observation, E::Action>)

Record a transition and update the agent’s internal state.

fn total_steps(&self) -> usize

Total number of observe calls since construction.

fn episode_extras(&self) -> HashMap<String, f64>

Per-episode aggregates of step-level values, reported at episode end.

fn on_episode_start(&mut self)

Called by crate::training::TrainingSession at the start of each episode so the agent can reset its per-episode aggregators.