Struct TimeLimit 

pub struct TimeLimit<E: Environment> { /* private fields */ }

Wraps an environment and truncates episodes after max_steps steps.

This is one of the most universally needed wrappers. Without it, environments without natural termination conditions (e.g. locomotion tasks) run forever.

Episodes truncated by this wrapper emit EpisodeStatus::Truncated, not EpisodeStatus::Terminated, so algorithms correctly bootstrap the value of the final state.

Implementations

impl<E: Environment> TimeLimit<E>

pub fn new(env: E, max_steps: usize) -> Self

Examples found in repository

examples/cartpole.rs (line 140)

fn main() {
    const NUM_EPISODES: usize = 10;
    const ENV_SEED: u64 = 42;

    let mut env = TimeLimit::new(CartPole::new(ENV_SEED), MAX_STEPS);
    let mut rng = SmallRng::seed_from_u64(0);

    println!("CartPole-v1 — random agent, {NUM_EPISODES} episodes\n");
    println!("{:<8} {:>8} {:>7} {:>12}", "Episode", "Return", "Steps", "Outcome");
    println!("{}", "-".repeat(40));

    let mut total_return = 0.0;

    for ep in 1..=NUM_EPISODES {
        let (ret, status, steps) = run_episode(&mut env, &mut rng);
        total_return += ret;

        let outcome = match status {
            EpisodeStatus::Terminated => "Terminated",
            EpisodeStatus::Truncated => "Truncated ",
            EpisodeStatus::Continuing => unreachable!(),
        };

        println!("{ep:<8} {ret:>8.1} {steps:>7} {outcome:>12}");
    }

    println!("{}", "-".repeat(40));
    println!(
        "Mean return over {NUM_EPISODES} episodes: {:.1}",
        total_return / NUM_EPISODES as f64
    );
}

pub fn elapsed_steps(&self) -> usize

pub fn remaining_steps(&self) -> usize

Trait Implementations

impl<E: Environment> Environment for TimeLimit<E>

type Observation = <E as Environment>::Observation

The observation type produced by step() and reset().

type Action = <E as Environment>::Action

The action type consumed by step().

type Info = <E as Environment>::Info

Auxiliary information returned alongside observations.

fn step( &mut self, action: Self::Action, ) -> StepResult<Self::Observation, Self::Info>

Advance the environment by one timestep.

fn reset(&mut self, seed: Option<u64>) -> (Self::Observation, Self::Info)

Reset the environment to an initial state, starting a new episode.

fn sample_action(&self, rng: &mut impl Rng) -> Self::Action

Sample a random action from this environment’s action space.

impl<E: Environment> Wrapper for TimeLimit<E>

type Inner = E

fn inner(&self) -> &E

fn inner_mut(&mut self) -> &mut E

fn unwrapped(&self) -> &Self::Inner

Unwrap all layers and return a reference to the base environment.