rlevo-environments

Standard benchmark environments and landscapes for the rlevo workspace.

This crate provides a collection of reinforcement learning environments ranging from classic tabular problems through continuous-control physics simulations. All environments implement the rlevo-core Environment trait — a common reset / step interface that makes them drop-in compatible with every algorithm in rlevo-deeprl.

Environments

Classic Control

Ports of the canonical Gymnasium control tasks implemented in pure Rust.

Toy Text

Tabular MDPs for baseline algorithm validation. Every state is fully observable.

Gridworlds

Twelve partially observable grid environments inspired by Farama Minigrid. All share a common egocentric 7×7×3 observation and a 7-action discrete space. Physics are implemented once in grids::core and reused across every variant.

Box2D-style Physics

Continuous-control environments powered by Rapier2D. Enabled by the box2d feature (default on).

Locomotion

MuJoCo-style locomotion environments in pure Rust via Rapier3D. Enabled by the locomotion feature (default on).

Games (planned for v0.2)

Chess and ConnectFour are planned for a future release. Stub modules exist in-source (src/games/chess/ and src/games/connect_four.rs) but do not yet implement the Environment trait and are hidden from the public API docs.

Optimization Landscapes

Continuous single-objective fitness functions for evaluating evolutionary algorithms.

Quick Start

use rlevo_core::environment::{Environment, Snapshot};
use rlevo_envs::classic::{CartPole, CartPoleConfig, CartPoleAction};
use rlevo_envs::wrappers::TimeLimit;

let env = CartPole::with_config(CartPoleConfig::default());
let mut env = TimeLimit::new(env, 500);

let mut snap = env.reset().expect("reset");
while !snap.is_done() {
    snap = env.step(CartPoleAction::Right).expect("step");
}

Gridworld environments use the shared GridAction type:

use rlevo_core::environment::{Environment, Snapshot};
use rlevo_envs::grids::{EmptyEnv, EmptyConfig};
use rlevo_envs::grids::core::GridAction;

let mut env = EmptyEnv::with_config(EmptyConfig::default(), false);
let mut snap = env.reset().expect("reset");
while !snap.is_done() {
    snap = env.step(GridAction::Forward).expect("step");
}

Cargo Features

Disable physics environments to shrink compile time:

[dependencies]
rlevo-environments = { path = "…", default-features = false }

Running Examples

# Classic control
cargo run -p rlevo-environments --example cartpole_random
cargo run -p rlevo-environments --example cartpole_timelimit
cargo run -p rlevo-environments --example pendulum_random
cargo run -p rlevo-environments --example mountain_car_random
cargo run -p rlevo-environments --example mountain_car_continuous_random
cargo run -p rlevo-environments --example acrobot_random

# Gridworlds
cargo run -p rlevo-environments --example grid_empty_random
cargo run -p rlevo-environments --example grid_door_key_scripted
cargo run -p rlevo-environments --example grid_memory_random

# Bandits
cargo run -p rlevo-environments --example ten_armed_bandit_training

# Box2D (requires box2d feature, enabled by default)
cargo run -p rlevo-environments --example bipedal_walker_random
cargo run -p rlevo-environments --example lunar_lander_discrete_random
cargo run -p rlevo-environments --example lunar_lander_continuous_random
cargo run -p rlevo-environments --example car_racing_random

# Locomotion (requires locomotion feature, enabled by default)
cargo run -p rlevo-environments --example reacher_random
cargo run -p rlevo-environments --example inverted_double_pendulum_random
cargo run -p rlevo-environments --example swimmer_random

See examples/README.md for patterns, conventions, and how to write new examples.

Design

Configuration Builder

Every environment ships with a XyzConfig struct and a XyzConfigBuilder with a fluent API. Defaults match the reference Gymnasium implementation where one exists.

use rlevo_environments::classic::{CartPole, CartPoleConfig};

let env = CartPole::with_config(
    CartPoleConfig {
        seed: 42,
        ..CartPoleConfig::default()
    }
);

Reproducibility

reset() re-seeds the environment's internal RNG from config.seed so the same seed always produces the same episode trajectory. Pass different seeds across parallel workers to get independent rollouts.

Wrappers

wrappers::TimeLimit<E> wraps any Environment and injects a truncation signal after a configurable number of steps. The underlying environment's Terminated status is preserved separately from the wrapper's Truncated — algorithms that distinguish the two (PPO, SAC) can act on both signals correctly.

Episode Status

Snapshot::status() returns one of three variants:

• Running — episode continues
• Terminated — natural episode end (goal reached, fell over, game over)
• Truncated — wall-clock time limit exceeded (TimeLimit wrapper)

Const Generics

State and observation dimensions are encoded as const generics (State<D>, Observation<D>). Mismatched dimensions produce a compile-time error rather than a runtime panic.

Testing

# Unit tests (all environments)
cargo test -p rlevo-environments

# Gridworld solvability integration tests
# (scripted optimal policies verify physics correctness)
cargo test -p rlevo-environments --test grids_solvable

References

• G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, "OpenAI Gym," arXiv preprint arXiv:1606.01540, Jun. 2016. https://arxiv.org/abs/1606.01540
• M. Chevalier-Boisvert, B. Dai, M. Towers, R. de Lazcano, L. Willems, S. Lahlou, S. Pal, P. S. Castro, and J. Terry, "Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks," arXiv preprint arXiv:2306.13831, Jun. 2023. https://arxiv.org/abs/2306.13831
• A. G. Barto, R. S. Sutton, and C. W. Anderson, "Neuronlike adaptive elements that can solve difficult learning control problems," IEEE Transactions on Systems, Man, and Cybernetics, vol. SMC-13, no. 5, pp. 834–846, Sep./Oct. 1983. doi: 10.1109/TSMC.1983.6313077.
• D. Silver, T. Hubert, J. Schrittwieser, I. Antonoglou, M. Lai, A. Guez, M. Lanctot, L. Sifre, D. Kumaran, T. Graepel, T. P. Lillicrap, K. Simonyan, and D. Hassabis, “Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm,” arXiv preprint arXiv:1712.01815, Dec. 2017. https://arxiv.org/abs/1712.01815
• J. Leike, M. Martic, V. Krakovna, P. A. Ortega, T. Everitt, A. Lefrancq, L. Orseau, and S. Legg, "AI safety gridworlds," arXiv preprint arXiv:1711.09883, Nov. 2017. https://arxiv.org/abs/1711.09883
• Rapier physics engine — https://rapier.rs
• Burn framework — https://burn.dev

License

Licensed under either of Apache License, Version 2.0 or MIT License at your option.