Struct QLearningSolver 

pub struct QLearningSolver {
    pub q: Vec<Vec<f64>>,
    pub alpha: f64,
    pub gamma: f64,
    pub epsilon: f64,
    pub visit_count: Vec<Vec<u64>>,
}

Q-learning solver with epsilon-greedy exploration and decaying step size.

Fields

q: Vec<Vec<f64>>

Q-value table Q[s][a].

alpha: f64

Learning rate α.

gamma: f64

Discount factor γ.

epsilon: f64

Exploration rate ε.

visit_count: Vec<Vec<u64>>

Step counter per (s,a) pair (for step-size decay).

Implementations

impl QLearningSolver

pub fn new( num_states: usize, num_actions: usize, alpha: f64, gamma: f64, epsilon: f64, ) -> Self

Construct a Q-learning solver.

pub fn update(&mut self, s: usize, a: usize, r: f64, s_next: usize)

Perform a Q-learning update with harmonic step size 1/(1 + n(s,a)).

pub fn select_action(&self, s: usize, rng_val: f64) -> usize

Select an action using epsilon-greedy policy (deterministic tie-breaking). rng_val ∈ [0,1) is a uniform random value supplied by the caller.

pub fn has_converged(&self, prev_q: &[Vec<f64>], tol: f64) -> bool

Check convergence: max |Q(s,a) - Q_prev(s,a)| < tol.

pub fn greedy_policy(&self) -> Vec<usize>

Return the current greedy policy.

Trait Implementations

impl Clone for QLearningSolver

fn clone(&self) -> QLearningSolver

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for QLearningSolver

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.