Struct relearn::envs::MemoryGame

pub struct MemoryGame {
    pub num_actions: usize,
    pub history_len: usize,
}

Memory Game Environment

The agent must remember the inital state and choose the corresponding action as the final action in an episode.

• The environment consists of (NUM_ACTIONS + HISTORY_LEN) states.
• An episode starts in a state [0, NUM_ACTIONS) uniformly at random.
• Step i in [0, HISTORY_LEN) transitions to state NUM_ACTIONS + i with 0 reward regardless of the action.
• On step HISTORY_LEN, the agent chooses one of NUM_ACTIONS actions and if the action index matches the index of the inital state then the agent earns +1 reward, otherwise it earns -1 reward. This step is terminal.
• Every episode has length HISTORY_LEN + 1.

Fields

num_actions: usize

The number of actions.

history_len: usize

Length of remembered history required to solve the environment.

Implementations

impl MemoryGame

pub const fn new(num_actions: usize, history_len: usize) -> Self

Create a new MemoryGame instance

Args

• num_actions - Number of possible actions.
• history_len - Length of remembered history required to solve the environment.

Trait Implementations

impl Clone for MemoryGame

fn clone(&self) -> MemoryGame

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for MemoryGame

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for MemoryGame

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for MemoryGame

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error> where
    __D: Deserializer<'de>, 

Deserialize this value from the given Serde deserializer.

impl EnvStructure for MemoryGame

type ObservationSpace = IndexSpace

type ActionSpace = IndexSpace

type FeedbackSpace = IntervalSpace<Reward>

fn observation_space(&self) -> Self::ObservationSpace

Space containing all possible observations.

fn action_space(&self) -> Self::ActionSpace

The space of all possible actions.

fn feedback_space(&self) -> Self::FeedbackSpace

The space of all possible feedback.

fn discount_factor(&self) -> f64

A discount factor applied to future feedback.

impl Environment for MemoryGame

type State = (usize, usize)

(current_state, initial_state)

type Observation = usize

Observation of the state provided to the agent.

type Action = usize

Action selected by the agent.

type Feedback = Reward

Feedback provided to a learning agent as the result of each step. Reward, for example.

fn initial_state(&self, rng: &mut Prng) -> Self::State

Sample a state for the start of a new episode.

fn observe(&self, state: &Self::State, _rng: &mut Prng) -> Self::Observation

Generate an observation for a given state.

fn step(
    &self,
    state: Self::State,
    action: &Self::Action,
    _: &mut Prng,
    _: &mut dyn StatsLogger
) -> (Successor<Self::State>, Self::Feedback)

Perform a state transition in reponse to an action.

fn run<T, L>(self, actor: T, seed: SimSeed, logger: L) -> Steps<Self, T, Prng, L> where
    T: Actor<Self::Observation, Self::Action>,
    L: StatsLogger,
    Self: Sized, 

Run this environment with the given actor.

impl Hash for MemoryGame

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H) where
    H: Hasher, 

Feeds a slice of this type into the given Hasher.

impl PartialEq<MemoryGame> for MemoryGame

fn eq(&self, other: &MemoryGame) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &MemoryGame) -> bool

This method tests for !=.

impl Serialize for MemoryGame

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error> where
    __S: Serializer, 

Serialize this value into the given Serde serializer.

impl CloneBuild for MemoryGame

impl Copy for MemoryGame

impl Eq for MemoryGame

impl StructuralEq for MemoryGame

impl StructuralPartialEq for MemoryGame