Struct QLearning 

pub struct QLearning {
    pub q_table: Array2<f64>,
    pub alpha: f64,
    pub epsilon: f64,
    pub gamma: f64,
}

Tabular Q-learning agent (model-free, off-policy TD).

Fields

q_table: Array2<f64>

Q-value table (n_states × n_actions).

alpha: f64

Learning rate α ∈ (0, 1].

epsilon: f64

ε-greedy exploration probability.

gamma: f64

Discount factor γ.

Implementations

impl QLearning

pub fn new( n_states: usize, n_actions: usize, alpha: f64, epsilon: f64, gamma: f64, ) -> Self

Create a new Q-learning agent with zero-initialised Q-table.

pub fn update( &mut self, state: usize, action: usize, reward: f64, next_state: usize, )

Apply a single Q-learning update.

Q(s,a) ← Q(s,a) + α [ r + γ max_{a'} Q(s',a') − Q(s,a) ]

pub fn epsilon_greedy(&self, state: usize, rng_seed: u64) -> usize

Select an action via ε-greedy policy (deterministic given rng_seed).

pub fn greedy(&self, state: usize) -> usize

Select the greedy action (no exploration).

pub fn train( &mut self, mdp: &Mdp, n_episodes: usize, max_steps_per_episode: usize, seed: u64, ) -> Result<Vec<f64>, OptimizeError>

Train Q-learning on a known MDP for n_episodes episodes.

Returns episode discounted returns.

pub fn policy(&self) -> Vec<usize>

Extract the greedy policy from Q-table.

pub fn value_function(&self) -> Vec<f64>

Estimate the value function: V(s) = max_a Q(s,a).

Trait Implementations

impl Clone for QLearning

fn clone(&self) -> QLearning

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for QLearning

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.