Module rollout 

Rollout buffer for storing and processing trajectories

This module implements experience storage for PPO training, including:

• Trajectory storage (observations, actions, rewards, etc.)
• Generalized Advantage Estimation (GAE) computation
• Efficient sampling for minibatch training

Buffer Layout

The buffer uses a [num_steps, num_envs] layout where:

• num_steps: Number of timesteps per rollout (typically 128-2048)
• num_envs: Number of parallel environments

This layout provides good cache locality for forward passes and efficient computation of advantages.

Structs

Minibatch

Minibatch data for training

MinibatchIterator

Iterator for generating minibatches from a rollout buffer

RecurrentMinibatchIterator

Env-major minibatch iterator over whole env-trajectories (full-sequence, Strategy A).

RecurrentRolloutBatch

A rank-3 batch of recurrent rollout data, ready to feed LstmBurnPolicy::evaluate_sequences with no shape adapters.

RecurrentRolloutBuffer

Rollout buffer that preserves temporal order for a recurrent policy.

RolloutBatch

Batch of rollout data for training

RolloutBuffer

Rollout buffer for storing trajectories

RolloutBurnTensors

Bundle of Burn tensors produced by RolloutBatch::to_burn_tensors.

Functions

compute_advantages

Compute Generalized Advantage Estimation (GAE)

compute_advantages_multi_agent

Compute Generalized Advantage Estimation (GAE) over a flat, interleaved multi-agent rollout buffer.

compute_advantages_partial

Compute Generalized Advantage Estimation (GAE) over the first valid_steps rows of the buffer.

compute_mc_returns

Compute Monte Carlo returns (full episode returns)

compute_nstep_returns

Compute n-step returns (simpler alternative to GAE)

compute_vtrace_advantages

Compute V-trace targets and advantages for all environments in the buffer.

generate_minibatch_indices

Generate minibatch indices for PPO training

normalize_advantages

Normalize advantages across the entire buffer

sample_minibatch

Sample a minibatch from the rollout buffer

shuffle_indices

Create a shuffled sequence of indices for experience replay

train_val_split

Split buffer into train/validation sets