Function compute_advantages 

pub fn compute_advantages(
    buffer: &mut RolloutBuffer,
    last_values: &[f32],
    gamma: f32,
    gae_lambda: f32,
)

Compute Generalized Advantage Estimation (GAE)

GAE computes advantages using a weighted sum of n-step returns, providing a balance between bias and variance.

Arguments

• buffer - Rollout buffer to compute advantages for
• last_values - Value estimates for the final states [num_envs]
• gamma - Discount factor (0 < gamma <= 1)
• gae_lambda - GAE lambda parameter (0 < lambda <= 1)

Mathematical Formula

δ_t = r_t + γ * V_{t+1} - V_t
A_t = δ_t + γ * λ * A_{t+1}

Where:

• δ_t is the temporal difference error
• A_t is the advantage estimate
• r_t is the reward at time t
• V_t is the value estimate at time t
• γ is the discount factor
• λ is the GAE parameter