Expand description

A reinforcement learning library.

This library defines a set of environments and learning agents and simulates their interaction.

Environments implement the Environment trait, which has associated observation, action, and state types. Agents implement Agent and provide Actors that generate actions in response to environment observations. Agents can learn via the BatchUpdate trait.

Agent traits are generic over the observation (O) and action (A) types of the environment. The EnvStructure trait provides more details about possible values for these types via the Space trait. A Space can be thought of as a runtime-defined type, describing a set of possible values while methods are provided by other traits in spaces.

Environment-actor simulation is performed by Steps and the resulting Step are accessible via an Iterator interface. Training is performed by train_serial and train_parallel.

This library uses PyTorch via tch.


pub use agents::Actor;
pub use agents::Agent;
pub use agents::BatchUpdate;
pub use agents::BuildAgent;
pub use envs::EnvStructure;
pub use envs::Environment;
pub use simulation::Simulation;
pub use simulation::Step;
pub use simulation::StepsIter;


Reinforcement learning agents

Reinforcement learning environments

Agent-environment feedback

Logging statistics from simulation runs

Simulating agent-environment interaction

Spaces: runtime-defined types

Torch components

General-purpose utilities


Iterator of environment-actor steps.



Train a batch learning agent in parallel across several threads.

Train a batch learning agent in this thread.

Type Definitions

Pseudo-random number generator type used by agents and environments in this crate.