Struct Actor

pub struct Actor<A, E, P, R>
where
    A: Agent<E, R> + Configurable + SyncModel + 'static,
    E: Env,
    P: StepProcessor<E>,
    R: ExperienceBufferBase<Item = P::Output> + ReplayBufferBase,
{ /* private fields */ }

Generate transitions by running Agent in Env.

flowchart TB E["Agent
(in AsyncTrainer)"]-->|SyncModel::ModelInfo|A subgraph D[Actor] A[Agent]-->|Env::Act|B[Env] B-->|Env::Obs|A B-->|Step<E: Env>|C[StepProcessor] end C-->|ReplayBufferBase::PushedItem|F[ReplayBufferProxy]

In Actor, an Agent runs on an Env and generates Step objects. These objects are processed with StepProcessor and sent to ReplayBufferProxy. The Agent in the Actor periodically synchronizes with the Agent in AsyncTrainer via SyncModel::ModelInfo.

See also the diagram in AsyncTrainer.

Implementations

impl<A, E, P, R> Actor<A, E, P, R>

where A: Agent<E, R> + Configurable + SyncModel + 'static, E: Env, P: StepProcessor<E>, R: ExperienceBufferBase<Item = P::Output> + ReplayBufferBase,

pub fn build( id: usize, agent_config: A::Config, env_config: E::Config, step_proc_config: P::Config, replay_buffer_config: ReplayBufferProxyConfig, stop: Arc<Mutex<bool>>, env_seed: i64, stats: Arc<Mutex<Option<ActorStat>>>, ) -> Self

pub fn run( &mut self, sender: Sender<PushedItemMessage<R::Item>>, model_info: Arc<Mutex<(usize, A::ModelInfo)>>, guard: Arc<Mutex<bool>>, guard_init_model: Arc<Mutex<bool>>, )

Runs sampling loop until self.stop becomes true.

When finishes, this method sets ActorStat.