Crate border_async_trainer

Asynchronous trainer using replay buffer.

Training process consists of the following two components:

• ActorManager manages Actors, each of which runs a thread for interacting Agent and Env and taking samples. Those samples will be sent to the replay buffer in AsyncTrainer.
• AsyncTrainer is responsible for training of an agent. It also runs a thread for pushing samples from ActorManager into a replay buffer.

The Agent must implement SyncModel trait in order to synchronize the model of the agent in Actor with the trained agent in AsyncTrainer. The trait has the ability to import and export the information of the model as SyncModel::ModelInfo.

The Agent in AsyncTrainer is responsible for training, typically with a GPU, while the Agents in Actors in ActorManager is responsible for sampling using CPU.

Both AsyncTrainer and ActorManager are running in the same machine and communicate by channels.

Structs

Actor

Runs interaction between an Agent and an Env, taking samples.

ActorManager

Manages Actors.

ActorManagerConfig

Configuration of ActorManager.

ActorStat

Stats of sampling process in each Actor.

AsyncTrainStat

Stats of AsyncTrainer::train().

AsyncTrainer

Manages asynchronous training loop in a single machine.

AsyncTrainerConfig

Configuration of AsyncTrainer

PushedItemMessage

Message containing a ReplayBufferBase::PushedItem.

ReplayBufferProxy

A wrapper of replay buffer for asynchronous trainer.

ReplayBufferProxyConfig

Configuration of ReplayBufferProxy.

Enums

BorderAsyncTrainerError

Traits

SyncModel

Synchronizes the model of the agent in asynchronous training.

Functions

actor_stats_fmt

Returns a formatted string of the set of ActorStat for reporting.