Struct ValueIteration 

pub struct ValueIteration {
    pub num_states: usize,
    pub num_actions: usize,
    pub transitions: Vec<Vec<Vec<f64>>>,
    pub rewards: Vec<Vec<f64>>,
    pub discount: f64,
}

Standalone discounted MDP value iteration solver.

More flexible than MDP::value_iteration — accepts sparse transition representations via closures (approximated here via table).

Fields

num_states: usize

Number of states.

num_actions: usize

Number of actions.

transitions: Vec<Vec<Vec<f64>>>

Transition probabilities T[s][a][s’].

rewards: Vec<Vec<f64>>

Reward R[s][a].

discount: f64

Discount factor γ.

Implementations

impl ValueIteration

pub fn new( num_states: usize, num_actions: usize, transitions: Vec<Vec<Vec<f64>>>, rewards: Vec<Vec<f64>>, discount: f64, ) -> Self

Construct a value iteration solver.

pub fn run(&self, tol: f64, max_iter: usize) -> (Vec<f64>, Vec<usize>)

Run value iteration and return (V*, π*).

pub fn q_from_v(&self, v: &[f64]) -> Vec<Vec<f64>>

Compute the Q-function from the optimal value function.

pub fn span(&self, v: &[f64]) -> f64

Span semi-norm: measure of Bellman residual (for convergence diagnostics).

Trait Implementations

impl Clone for ValueIteration

fn clone(&self) -> ValueIteration

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ValueIteration

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.