Struct Sarsa 

pub struct Sarsa {
    pub q_table: Array2<f64>,
    pub alpha: f64,
    pub epsilon: f64,
    pub gamma: f64,
}

Tabular SARSA agent (on-policy TD learning).

Fields

q_table: Array2<f64>

Q-value table (n_states × n_actions).

alpha: f64

Learning rate.

epsilon: f64

ε-greedy exploration rate.

gamma: f64

Discount factor.

Implementations

impl Sarsa

pub fn new( n_states: usize, n_actions: usize, alpha: f64, epsilon: f64, gamma: f64, ) -> Self

Create a new SARSA agent.

pub fn update( &mut self, s: usize, a: usize, r: f64, s_next: usize, a_next: usize, )

Apply one SARSA TD update.

Q(s,a) ← Q(s,a) + α [ r + γ Q(s',a') − Q(s,a) ]

pub fn train( &mut self, mdp: &Mdp, n_episodes: usize, max_steps: usize, seed: u64, ) -> Result<Vec<f64>, OptimizeError>

Train SARSA on an MDP for n_episodes episodes.

pub fn policy(&self) -> Vec<usize>

Extract greedy policy.

Trait Implementations

impl Clone for Sarsa

fn clone(&self) -> Sarsa

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Sarsa

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.