Struct relearn::envs::Chain

pub struct Chain {
    pub size: usize,
    pub discount_factor: f64,
}

Chain Environment

Consists of n states in a line with 2 actions.

• Action 0 moves back to the start for 2 reward.
• Action 1 moves forward for 0 reward in all states but the last. In the last state, taking action 1 is a self-transition with 10 reward.
• Every action has a 0.2 chance of “slipping” and taking the opposite action.

Described in “Bayesian Q-learning” by Dearden, Friedman and Russel (1998)

Fields

size: usize

discount_factor: f64

Implementations

impl Chain

pub const fn new(size: usize, discount_factor: f64) -> Self

Trait Implementations

impl Clone for Chain

fn clone(&self) -> Chain

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Chain

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Chain

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for Chain

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error> where
    __D: Deserializer<'de>, 

Deserialize this value from the given Serde deserializer.

impl EnvStructure for Chain

type ObservationSpace = IndexSpace

type ActionSpace = IndexedTypeSpace<Move>

type FeedbackSpace = IntervalSpace<Reward>

fn observation_space(&self) -> Self::ObservationSpace

Space containing all possible observations.

fn action_space(&self) -> Self::ActionSpace

The space of all possible actions.

fn feedback_space(&self) -> Self::FeedbackSpace

The space of all possible feedback.

fn discount_factor(&self) -> f64

A discount factor applied to future feedback.

impl Environment for Chain

type State = usize

Environment state type. Not necessarily observable by the agent.

type Observation = usize

Observation of the state provided to the agent.

type Action = Move

Action selected by the agent.

type Feedback = Reward

Feedback provided to a learning agent as the result of each step. Reward, for example.

fn initial_state(&self, _: &mut Prng) -> Self::State

Sample a state for the start of a new episode.

fn observe(&self, state: &Self::State, _: &mut Prng) -> Self::Observation

Generate an observation for a given state.

fn step(
    &self,
    state: Self::State,
    action: &Self::Action,
    rng: &mut Prng,
    _: &mut dyn StatsLogger
) -> (Successor<Self::State>, Self::Feedback)

Perform a state transition in reponse to an action.

fn run<T, L>(self, actor: T, seed: SimSeed, logger: L) -> Steps<Self, T, Prng, L> where
    T: Actor<Self::Observation, Self::Action>,
    L: StatsLogger,
    Self: Sized, 

Run this environment with the given actor.

impl PartialEq<Chain> for Chain

fn eq(&self, other: &Chain) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Chain) -> bool

This method tests for !=.

impl Serialize for Chain

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error> where
    __S: Serializer, 

Serialize this value into the given Serde serializer.

impl CloneBuild for Chain

impl Copy for Chain

impl StructuralPartialEq for Chain