Struct Chain

pub struct Chain {
    pub size: usize,
    pub discount_factor: f64,
}

Chain Environment

Consists of n states in a line with 2 actions.

• Action 0 moves back to the start for 2 reward.
• Action 1 moves forward for 0 reward in all states but the last. In the last state, taking action 1 is a self-transition with 10 reward.
• Every action has a 0.2 chance of “slipping” and taking the opposite action.

Described in “Bayesian Q-learning” by Dearden, Friedman and Russel (1998)

Fields

size: usize

discount_factor: f64

Implementations

impl Chain

pub const fn new(size: usize, discount_factor: f64) -> Self

Trait Implementations

impl Clone for Chain

fn clone(&self) -> Chain

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Chain

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Chain

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for Chain

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl EnvStructure for Chain

type ObservationSpace = IndexSpace

type ActionSpace = IndexedTypeSpace<Move>

type FeedbackSpace = IntervalSpace<Reward>

fn observation_space(&self) -> Self::ObservationSpace

Space containing all possible observations.

fn action_space(&self) -> Self::ActionSpace

The space of all possible actions.

fn feedback_space(&self) -> Self::FeedbackSpace

The space of all possible feedback.

fn discount_factor(&self) -> f64

A discount factor applied to future feedback.

impl Environment for Chain

type State = usize

Environment state type. Not necessarily observable by the agent.

type Observation = usize

Observation of the state provided to the agent.

type Action = Move

Action selected by the agent.

type Feedback = Reward

Feedback provided to a learning agent as the result of each step. Reward, for example.

fn initial_state(&self, _: &mut Prng) -> Self::State

Sample a state for the start of a new episode.

fn observe(&self, state: &Self::State, _: &mut Prng) -> Self::Observation

Generate an observation for a given state.

fn step( &self, state: Self::State, action: &Self::Action, rng: &mut Prng, _: &mut dyn StatsLogger, ) -> (Successor<Self::State>, Self::Feedback)

Perform a state transition in reponse to an action.

fn run<T, L>( self, actor: T, seed: SimSeed, logger: L, ) -> Steps<Self, T, Prng, L>

where T: Actor<Self::Observation, Self::Action>, L: StatsLogger, Self: Sized,

Run this environment with the given actor.

impl PartialEq for Chain

fn eq(&self, other: &Chain) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for Chain

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl CloneBuild for Chain

impl Copy for Chain

impl StructuralPartialEq for Chain