Trait relearn::agents::BatchUpdate

source · [−]
pub trait BatchUpdate<O, A> {
    type Feedback;
    type HistoryBuffer: WriteExperience<O, A, Self::Feedback>;

    fn buffer(&self) -> Self::HistoryBuffer;
    fn min_update_size(&self) -> HistoryDataBound;
    fn batch_update<'a, I>(&mut self, buffers: I, logger: &mut dyn StatsLogger)
    where
        I: IntoIterator<Item = &'a mut Self::HistoryBuffer>,
        Self::HistoryBuffer: 'a;
}
Expand description

Update an agent with steps collected into history buffers.

The user is only responsible for adding data to the history buffer. If old data in the buffer needs to be cleared that is the reponsibility of either the buffer or the batch_update method.

Required Associated Types

Environment feedback type

Required Methods

Create a new history buffer.

Request a minimum amount of on-policy data for the next batch update.

Update the agent from a collection of history buffers.

All new data inserted into the buffers since the last call must be on-policy.

It is preferable but not required for the last step of the newly added data to end its episode — either by termination or interruption. If instead the last step.next is Successor::Continue then that one step may be dropped or ignored by agents that depend on well-formed step successors.

This function and the history buffer itself are jointly reponsible for managing the data within the buffer. The buffers may be emptied by the call.

Implementation Note

This iterator-generic batch_update is necessary because some agents modify the given buffer references before passing to batch_update of an inner agent. This would not work if the inner agent only took a slice of buffers.

Implementations on Foreign Types

No updates at the meta-level.

Implementors