Struct PolicyGradient 

pub struct PolicyGradient {
    pub theta: Vec<Vec<f64>>,
    pub alpha: f64,
    pub gamma: f64,
}

Policy gradient agent (softmax parameterisation).

Policy: π_θ(a|s) = exp(θ[s][a]) / Σ exp(θ[s][a’]) Update: θ[s][a] += α · ∇_θ log π_θ(a|s) · G where G is the return.

Fields

theta: Vec<Vec<f64>>

Policy parameter table θ[s][a].

alpha: f64

Learning rate α.

gamma: f64

Discount factor γ.

Implementations

impl PolicyGradient

pub fn new( num_states: usize, num_actions: usize, alpha: f64, gamma: f64, ) -> Self

Construct a policy gradient agent.

pub fn softmax(&self, s: usize) -> Vec<f64>

Compute the softmax policy π_θ(·|s).

pub fn update(&mut self, s: usize, a: usize, g: f64)

Update θ using a single (s, a, G) sample from a trajectory.

pub fn expected_return(&self, s: usize, q: &ActionValueFunction) -> f64

Expected return from state s as E_π[Q(s,a)].

pub fn convergence_rate(&self, q: &ActionValueFunction) -> f64

Convergence rate estimate: max |∇J| across states (gradient norm).

Trait Implementations

impl Clone for PolicyGradient

fn clone(&self) -> PolicyGradient

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PolicyGradient

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.