Struct relearn::torch::critic::Gae

pub struct Gae<V> {
    pub gamma: f64,
    pub lambda: f64,
    pub value_fn: V,
}

Generalized Advantage Estimator critic.

Note

Currently does not properly handle non-terminal end-of-episode. This assumes that all episodes end with a reward of 0.

Reference

High-Dimensional Continuous Control Using Generalized Advantage Estimation. ICLR 2016 by John Schulman, Philipp Moritz, Sergey Levine, Michael I. Jordan, Pieter Abbeel https://arxiv.org/pdf/1506.02438.pdf

Fields

gamma: f64

Clips the environment discount factor to be no more than this.

lambda: f64

Advantage interpolation factor between one-step residuals (=0) and full return (=1).

value_fn: V

State value function module.

Trait Implementations

impl<V: Clone> Clone for Gae<V>

fn clone(&self) -> Gae<V>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<V> Critic for Gae<V> where
    V: SequenceModule, 

fn trainable(&self) -> bool

Whether this critic has trainable internal parameters

fn discount_factor(&self, env_discount_factor: f64) -> f64

Get the discount factor to use when calculating step returns.

fn seq_packed(&self, features: &dyn PackedHistoryFeaturesView) -> Tensor

Provide values for a packed sequence of steps.

fn loss(&self, features: &dyn PackedHistoryFeaturesView) -> Option<Tensor>

The loss of any trainable internal variables given the observed history features.

impl<V: Debug> Debug for Gae<V>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<V: PartialEq> PartialEq<Gae<V>> for Gae<V>

fn eq(&self, other: &Gae<V>) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Gae<V>) -> bool

This method tests for !=.

impl<V: Copy> Copy for Gae<V>

impl<V> StructuralPartialEq for Gae<V>