Struct Mdp 

pub struct Mdp {
    pub n_states: usize,
    pub n_actions: usize,
    pub transition: Array3<f64>,
    pub reward: Array3<f64>,
    pub gamma: f64,
    pub terminal_states: Vec<usize>,
}

A finite Markov Decision Process.

Transition probabilities: T[s, a, s'] = P(s' | s, a).
Rewards: R[s, a, s'] (triple-index form; use Mdp::with_state_action_reward for 2-D rewards).

Fields

n_states: usize

Number of states.

n_actions: usize

Number of actions.

transition: Array3<f64>

Transition tensor (n_states × n_actions × n_states).

reward: Array3<f64>

Reward tensor (n_states × n_actions × n_states).

gamma: f64

Discount factor γ ∈ [0, 1).

terminal_states: Vec<usize>

Optional absorbing / terminal states (no transitions away).

Implementations

impl Mdp

pub fn new( n_states: usize, n_actions: usize, transition: Array3<f64>, reward: Array3<f64>, gamma: f64, ) -> Result<Self, OptimizeError>

Create a new MDP and validate it.

pub fn validate(&self) -> Result<(), OptimizeError>

Validate that transition probabilities sum to 1 for each (s, a).

pub fn expected_reward(&self) -> Array2<f64>

Expected reward R(s,a) = Σ_{s’} T(s,a,s’) · R(s,a,s’).

pub fn with_state_action_reward( n_states: usize, n_actions: usize, transition: Array3<f64>, reward: Array2<f64>, gamma: f64, ) -> Result<Self, OptimizeError>

Build an MDP from a 2-D reward matrix (state × action) by broadcasting to 3-D.

Trait Implementations

impl Clone for Mdp

fn clone(&self) -> Mdp

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Mdp

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.