Struct PPOConfig 

pub struct PPOConfig {
    pub learning_rate: f64,
    pub n_epochs: usize,
    pub batch_size: usize,
    pub gamma: f64,
    pub gae_lambda: f64,
    pub clip_range: f64,
    pub clip_range_vf: f64,
    pub vf_coef: f64,
    pub ent_coef: f64,
    pub max_grad_norm: f64,
    pub target_kl: f64,
    pub seed: u64,
}

PPO configuration parameters

These hyperparameters control the PPO training process. Default values are based on common settings that work well for simple environments like CartPole.

Fields

learning_rate: f64

Learning rate for policy and value function

n_epochs: usize

Number of training epochs per rollout

batch_size: usize

Minibatch size for training

gamma: f64

Discount factor (gamma)

gae_lambda: f64

GAE lambda parameter

clip_range: f64

PPO clipping parameter (epsilon)

clip_range_vf: f64

Value function clipping parameter

vf_coef: f64

Value function loss coefficient

ent_coef: f64

Entropy bonus coefficient

max_grad_norm: f64

Maximum gradient norm for clipping.

Applied by both PPOTrainerBurn and RecurrentPPOTrainer as a global L2-norm clip (see clip_grads_by_global_norm) over the concatenation of every parameter’s gradient, immediately before each optimizer step (issue #299; prior to that fix the field was silently ignored by both trainers). Must be positive — validate rejects non-positive values — so to effectively disable clipping set a very large cap (e.g. 1e9).

target_kl: f64

Target KL divergence for early stopping

seed: u64

Seed for the inner-loop RNG used to shuffle minibatch indices.

Plumbed through the trainer so that two trainers built with the same seed produce bit-identical updates on the same rollout (issue #109). The PPO trainer constructs an internal StdRng::seed_from_u64(seed) at new() time and uses it for every minibatch shuffle.

Implementations

impl PPOConfig

pub fn new() -> Self

Create a new default configuration

pub fn validate(&self) -> Result<()>

Validate configuration parameters

pub fn learning_rate(self, lr: f64) -> Self

Set learning rate

pub fn n_epochs(self, epochs: usize) -> Self

Set number of training epochs

pub fn batch_size(self, size: usize) -> Self

Set minibatch size

pub fn gamma(self, gamma: f64) -> Self

Set discount factor

pub fn gae_lambda(self, lambda: f64) -> Self

Set GAE lambda

pub fn clip_range(self, clip: f64) -> Self

Set PPO clipping parameter

pub fn clip_range_vf(self, clip: f64) -> Self

Set value function clipping parameter

pub fn vf_coef(self, coef: f64) -> Self

Set value function loss coefficient

pub fn ent_coef(self, coef: f64) -> Self

Set entropy bonus coefficient

pub fn max_grad_norm(self, norm: f64) -> Self

Set maximum gradient norm

pub fn target_kl(self, kl: f64) -> Self

Set target KL divergence

pub fn seed(self, seed: u64) -> Self

Set the seed for the inner-loop RNG used for minibatch shuffling.

See PPOConfig::seed.

Trait Implementations

impl Clone for PPOConfig

fn clone(&self) -> PPOConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PPOConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for PPOConfig

fn default() -> Self

Returns the “default value” for a type.