Crate border

Border is a library for reinforcement learning (RL). The aim of the library is to provide both environment- and agent-agnostic framework based on the expressiveness and type-safety of Rust. In order to do this, border provides a function for evaluation (eval) and a trait for training (Trainer). These function and trait use some traits consisting main components in RL such as observation (Obs), action (Act), and environment (Env). It should be noted that in border, Policy represents controllers, which interacts with environments, while Agent provides the interface to train policies by working on Trainer.

Using these components, border implements build-in environments and RL algorithms. PyGymEnv and PyVecGymEnv are wrapper around OpenAI gym implemented in Python. These wrappers allow the library to use a lot of environments, including atari 2600 and pybullet-gym, commonly used in the RL comminity.

Border has some RL agents for discrete action (DQN) and continuous action (SAC). These agents use tch, a wrapper of libtorch.

Border enables to make environments and agents be independend regarding their internal computation. Actually, PyGymEnv and PyVecGymEnv use ndarray in order to represent observation and action. In contrast, RL agents mentioned above use tch::Tensor for internal computation. Functions and traits in env::py_gym_env::tch bridges the gap between these environments and agents. Like this, border is flexible in combination of environments and agents. By providing some bridges, it is able to switch computational backends or internal implementation of environments and agents. This concept is illustrated in schematic diagram of eval and Trainer.

Re-exports

pub use crate::core::util::eval;

pub use crate::core::trainer::Trainer;

pub use crate::core::base::Act;

pub use crate::core::base::Obs;

pub use crate::core::base::Env;

pub use crate::core::base::Policy;

pub use crate::core::base::Agent;

pub use crate::env::py_gym_env::PyGymEnv;

pub use crate::env::py_gym_env::PyVecGymEnv;

pub use crate::agent::tch::DQN;

pub use crate::agent::tch::SAC;

Modules

agent	Agents of reinforcement learning.
core	Primitives in reinforcement learning.
env	Environments.
error	Errors in the library.