Module env 

Environment traits and implementations Environment traits and implementations

This module defines the core environment interface and provides built-in environments for reinforcement learning.

Action types and continuous (Box) action spaces

The Environment trait exposes its action type as an associated type Environment::Action. Discrete envs (CartPole, Snake, Pong, SimpleBandit, BucketBrigade) set type Action = i64;. Continuous envs set type Action = Vec<f32>; (or another float type for single-dim cases). See games::continuous_lqr::ContinuousLqr for a minimal continuous-action existence proof.

Why the associated-type strategy (Strategy A)?

When this trait was extended to support continuous-control algorithms (SAC, TD3, DDPG, PPO with Gaussian heads), there were two candidate designs:

• A. Associated Action type on Environment (chosen). One trait, parameterised by the action type. Discrete envs default to type Action = i64; so existing call sites and trainers migrate by adding a single line per env impl.
• B. Parallel ContinuousEnvironment trait. Two distinct traits; envs implement one or the other (or both). The trainer dispatches per-trait.

Strategy A was picked because:

1. One trait. Downstream code (snapshot/restore, pool, multi-agent simulator) does not have to branch on which trait an env implements. There is exactly one Environment to reason about.
2. Cheap default for the common case. Every discrete env in the repo today gets type Action = i64; as a one-line addition. Continuous envs opt in to Vec<f32>.
3. Orthogonal to other trait extensions. The associated-type approach composes cleanly with planned additions like a type State slot for env snapshotting (issue #62 / clone_state / restore_state). Strategy B would have forced snapshot machinery to be implemented twice or behind a third trait.
4. Generic trainers stay generic. Discrete trainers add a <Action = i64> bound on their E: Environment type parameter, which is purely additive.

The trade-off: any code that wants to handle both discrete and continuous actions polymorphically must either be parametric on E::Action or use a sum-type wrapper. Until SAC lands, no such call site exists in Thrust.

Env state snapshots (type State / clone_state / restore_state)

The Environment trait also exposes a type State associated type plus clone_state / restore_state methods so callers (MCTS-style search, replay tooling) can snapshot env state and roll out hypothetical trajectories without restarting from reset(). The determinism contract is per-env: fully deterministic envs (e.g. CartPole, ContinuousLqr) reproduce every subsequent step bit-for-bit after restore; envs that consume internal RNG (Snake food respawn, Pong ball serve) snapshot the simulation step but not the RNG, so reproduction is deterministic only across steps that do not draw from the RNG. Per-env docs spell out the exact guarantee.

Re-exports

pub use games::CartPole;

pub use games::ContinuousLqr;

pub use games::FlickeringCartPole;

pub use games::GridWorld;

pub use games::MaskedCartPole;

pub use games::MountainCarContinuous;

pub use games::PendulumSwingUp;

pub use games::Pong;

pub use games::SimpleBandit;

pub use games::SnakeEnv;

pub use games::TMaze;

pub use games::cartpole;

pub use games::continuous_lqr;

pub use games::flickering_cartpole;

pub use games::grid_world;

pub use games::masked_cartpole;

pub use games::mountain_car_continuous;

pub use games::pendulum;

pub use games::pong;

pub use games::simple_bandit;

pub use games::snake;

pub use games::t_maze;

pub use games::SignalingGame;

pub use games::signaling;

Modules

games

Game environment implementations

pool

Vectorized environment pool for parallel execution

Structs

SpaceInfo

Space information for observations and actions

StepInfo

Additional step information

StepResult

Result of an environment step

Enums

SpaceType

Space data types

Traits

Environment

Core trait for RL environments