Struct gym_rs::envs::classical_control::cartpole::CartPoleEnv

pub struct CartPoleEnv {

    pub action_space: Discrete,
    pub observation_space: BoxR<CartPoleObservation>,
    pub render_mode: RenderMode,
    pub state: CartPoleObservation,
    pub metadata: Metadata<Self>,
    pub gravity: O64,
    pub masscart: O64,
    pub masspole: O64,
    pub length: O64,
    pub force_mag: O64,
    pub tau: O64,
    pub kinematics_integrator: KinematicsIntegrator,
    pub theta_threshold_radians: O64,
    pub x_threshold: O64,
    pub steps_beyond_terminated: Option<usize>,
    /* private fields */
}

An environment which implements the cart pole problem described in Neuronlike adaptive elements that can solve difficult learning control problems.

The problem involves applying the correct forces onto a cart with a pole hinged onto it, in order to ensure the pole remains within the preconfigured regions. The agent starts by being assigned random values between (-0.05, 0.05) for all fields available in the state structure. The agent is rewarded ‘+1’ for every step taken until the episode ends.

The episode ends when any of the following conditions occur:

1. Termination: [CartPoleObservation.theta] is greater than +/-12.0 (pole has fallen).
2. Termination: [CartPoleObservation.x] is greater than +/-2.4 (cart is outside bounds).
3. Truncation: Episode length is greater than 500.

Fields

action_space: Discrete

The available actions that can be taken.

observation_space: BoxR<CartPoleObservation>

The range of values that can be observed.

render_mode: RenderMode

The type of renders produced.

state: CartPoleObservation

The current state of the environment.

metadata: Metadata<Self>

Additional pieces of information provided by the environment.

gravity: O64

The gravity constant applied to the environment..

masscart: O64

The mass of the cart.

masspole: O64

The mass of the pole.

length: O64

Half the length of the pole.

force_mag: O64

The default force applied to the pole.

tau: O64

The number of seconds between state updates.

kinematics_integrator: KinematicsIntegrator

The type of integration done on the differential equations found in the paper.

theta_threshold_radians: O64

The angle that the pole can lean to before an episode is considered terminated.

x_threshold: O64

The x value that the cart can be at before an episode is considered terminated.

steps_beyond_terminated: Option<usize>

The number of steps taken after the episode was terminated.

Implementations

impl CartPoleEnv

pub fn new(render_mode: RenderMode) -> Self

Creates a cart pole environment using defaults from the paper.

Examples found in repository

examples/cartpole.rs (line 8)

fn main() {
    let mut env = CartPoleEnv::new(RenderMode::Human);
    env.reset(None, false, None);

    let mut rewards = vec![];

    for _ in 0..15 {
        let mut current_reward = OrderedFloat(0.);

        for _ in 0..475 {
            let action = (&mut thread_rng()).gen_range(0..=1);
            let state_reward = env.step(action);
            current_reward += state_reward.reward;

            if state_reward.done {
                break;
            }
        }

        env.reset(None, false, None);
        rewards.push(current_reward);
    }

    println!("{:?}", rewards);
}

Trait Implementations

impl Clone for CartPoleEnv

fn clone(&self) -> CartPoleEnv

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for CartPoleEnv

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Env for CartPoleEnv

type Action = usize

The type of action supported.

type Observation = CartPoleObservation

The type of the observation produced after an action has been applied.

type Info = ()

The type of the metadata object produced by acting on the environment.

type ResetInfo = ()

The type of the object produced when an environment is reset.

fn step( &mut self, action: Self::Action ) -> ActionReward<Self::Observation, Self::Info>

Acts on an environment using the given action, producing a reward.

fn reset( &mut self, seed: Option<u64>, return_info: bool, options: Option<BoxR<Self::Observation>> ) -> (Self::Observation, Option<Self::ResetInfo>)

Resets the environment to a initial random state.

fn render(&mut self, mode: RenderMode) -> Renders

Produces the renders, if any, associated with the given mode.

fn close(&mut self)

Closes any open resources associated with the internal rendering service.

impl EnvProperties for CartPoleEnv

type ActionSpace = Discrete

The type of values that can be observed in the action space.

type ObservationSpace = BoxR<CartPoleObservation>

The type of observations produced

fn metadata(&self) -> &Metadata<Self>

Provides an object describing additional details about this environment.

fn rand_random(&self) -> &Pcg64

Provides the random number generator responsible for seeding states.

fn action_space(&self) -> &Self::ActionSpace

Provides the object describing the actions that can be observed.

fn observation_space(&self) -> &Self::ObservationSpace

Provides the object describing the states that can be observed in this environment.

fn render_mode(&self) -> &RenderMode

Provides the current render mode.

fn reward_range(&self) -> &RewardRange

Provides the range of reward values that can be outputted by this environment.

impl Serialize for CartPoleEnv

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>where __S: Serializer,

Serialize this value into the given Serde serializer.