Struct relearn::envs::MemoryGame

pub struct MemoryGame {
    pub num_actions: usize,
    pub history_len: usize,
}

Memory Game Environment

The agent must remember the inital state and choose the corresponding action as the final action in an episode.

• The environment consists of (NUM_ACTIONS + HISTORY_LEN) states.
• An episode starts in a state [0, NUM_ACTIONS) uniformly at random.
• Step i in [0, HISTORY_LEN) transitions to state NUM_ACTIONS + i with 0 reward regardless of the action.
• On step HISTORY_LEN, the agent chooses one of NUM_ACTIONS actions and if the action index matches the index of the inital state then the agent earns +1 reward, otherwise it earns -1 reward. This step is terminal.
• Every episode has length HISTORY_LEN + 1.

Fields

num_actions: usize

The number of actions.

history_len: usize

Length of remembered history required to solve the environment.

Implementations

impl MemoryGame

pub const fn new(num_actions: usize, history_len: usize) -> Self

Create a new MemoryGame instance

Args

• num_actions - Number of possible actions.
• history_len - Length of remembered history required to solve the environment.

Trait Implementations

impl Clone for MemoryGame

fn clone(&self) -> MemoryGame

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for MemoryGame

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for MemoryGame

fn default() -> Self

Returns the “default value” for a type.

impl EnvStructure for MemoryGame

type ObservationSpace = IndexSpace

type ActionSpace = IndexSpace

fn observation_space(&self) -> Self::ObservationSpace

Space containing all possible observations.

fn action_space(&self) -> Self::ActionSpace

The space of all possible actions.

fn reward_range(&self) -> (f64, f64)

A lower and upper bound on possible reward values.

fn discount_factor(&self) -> f64

A discount factor applied to future rewards.

impl From<&'_ Options> for MemoryGame

fn from(opts: &Options) -> Self

Performs the conversion.

impl Hash for MemoryGame

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H) where
    H: Hasher, 

Feeds a slice of this type into the given Hasher.

impl PartialEq<MemoryGame> for MemoryGame

fn eq(&self, other: &MemoryGame) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &MemoryGame) -> bool

This method tests for !=.

impl Pomdp for MemoryGame

type State = (usize, usize)

(current_state, initial_state)

type Observation = usize

type Action = usize

fn initial_state(&self, rng: &mut StdRng) -> Self::State

Sample a new initial state.

fn observe(&self, state: &Self::State, _rng: &mut StdRng) -> Self::Observation

Sample an observation for a state.

fn step(
    &self,
    state: Self::State,
    action: &Self::Action,
    _rng: &mut StdRng
) -> (Option<Self::State>, f64, bool)

Sample a state transition.

impl Update<&'_ Options> for MemoryGame

fn update(&mut self, opts: &Options)

Update in-place from the given source value.

impl CloneBuild for MemoryGame

impl Copy for MemoryGame

impl Eq for MemoryGame

impl StructuralEq for MemoryGame

impl StructuralPartialEq for MemoryGame