Struct GenericTransitionBatch

pub struct GenericTransitionBatch<O, A>
where
    O: BatchBase,
    A: BatchBase,
{
    pub obs: O,
    pub act: A,
    pub next_obs: O,
    pub reward: Vec<f32>,
    pub is_terminated: Vec<i8>,
    pub is_truncated: Vec<i8>,
    pub weight: Option<Vec<f32>>,
    pub ix_sample: Option<Vec<usize>>,
}

A generic structure representing transitions in reinforcement learning.

This structure efficiently manages reinforcement learning transitions (observations, actions, rewards, etc.). It also includes support for prioritized experience replay (PER).

Type Parameters

• O - Observation type, must implement BatchBase
• A - Action type, must implement BatchBase

Examples

let batch = GenericTransitionBatch::<Tensor, Tensor>::with_capacity(32);

Fields

obs: O

Current observations

act: A

Selected actions

next_obs: O

Next state observations

reward: Vec<f32>

Transition rewards

is_terminated: Vec<i8>

Episode termination flags

is_truncated: Vec<i8>

Episode truncation flags

weight: Option<Vec<f32>>

Weights for prioritized experience replay

ix_sample: Option<Vec<usize>>

Indices of sampled transitions

Implementations

impl<O, A> GenericTransitionBatch<O, A>

where O: BatchBase, A: BatchBase,

pub fn with_capacity(capacity: usize) -> Self

Creates a new batch with the specified capacity.

Arguments

• capacity - Initial capacity of the batch

Returns

A new GenericTransitionBatch instance

Trait Implementations

impl<O, A> TransitionBatch for GenericTransitionBatch<O, A>

where O: BatchBase, A: BatchBase,

fn unpack( self, ) -> (Self::ObsBatch, Self::ActBatch, Self::ObsBatch, Vec<f32>, Vec<i8>, Vec<i8>, Option<Vec<usize>>, Option<Vec<f32>>)

Decomposes the batch into its individual components.

Returns

A tuple containing the following elements:

1. Observations
2. Actions
3. Next observations
4. Rewards
5. Termination flags
6. Truncation flags
7. Sample indices
8. Weights

fn len(&self) -> usize

Returns the number of transitions in the batch.

fn obs(&self) -> &Self::ObsBatch

Returns a reference to the batch of observations.

fn act(&self) -> &Self::ActBatch

Returns a reference to the batch of actions.

type ObsBatch = O

The type used to store batches of observations.

type ActBatch = A

The type used to store batches of actions.