Struct PrioritizedReplayBuffer 

pub struct PrioritizedReplayBuffer { /* private fields */ }

Fixed-capacity prioritized replay buffer.

Stores (obs, action, reward, next_obs, done) transitions in a ring buffer (same FIFO eviction as super::ReplayBuffer), plus a per-transition priority in a SumTree for O(log N) proportional sampling.

New transitions are pushed with the maximum existing priority (or 1.0 for an empty buffer), guaranteeing each fresh transition is sampled at least once before its priority can be updated by a TD error.

Implementations

impl PrioritizedReplayBuffer

pub fn new(capacity: usize, obs_dim: usize, alpha: f32, epsilon: f32) -> Self

Create a new prioritized replay buffer.

Arguments

• capacity - Maximum number of transitions stored. Must be > 0.
• obs_dim - Length of one observation vector. Must be > 0.
• alpha - Priority exponent α ∈ [0, 1]. 0 recovers uniform sampling; 1 is fully proportional. Typical value 0.6.
• epsilon - Tiny constant added to |TD error| before raising to α, so transitions with zero TD error still have a tiny chance of being resampled. Typical value 1e-6.

Panics

Panics if capacity == 0, obs_dim == 0, alpha is outside [0, 1], or epsilon < 0.

pub fn push( &mut self, obs: &[f32], action: i64, reward: f32, next_obs: &[f32], done: bool, )

Push a single transition into the buffer with the current maximum priority (or 1.0 if the buffer is empty).

When the buffer is at capacity, this overwrites the oldest transition (FIFO eviction); the corresponding leaf in the sum-tree is overwritten with the new priority.

pub fn len(&self) -> usize

Number of transitions currently in the buffer.

pub fn is_empty(&self) -> bool

true if the buffer holds no transitions.

pub fn capacity(&self) -> usize

Maximum number of transitions the buffer can hold.

pub fn obs_dim(&self) -> usize

Observation dimension (length of one obs slice).

pub fn alpha(&self) -> f32

Priority exponent α.

pub fn epsilon(&self) -> f32

Priority floor ε.

pub fn max_priority(&self) -> f32

Maximum leaf priority currently in the tree.

Useful for diagnostics; the buffer also tracks this internally so push can hand fresh transitions a max-priority slot.

pub fn is_ready(&self, min_size: usize) -> bool

true if the buffer holds at least min_size transitions.

pub fn sample<R: Rng>( &self, batch_size: usize, beta: f32, rng: &mut R, ) -> PrioritizedBatch

Sample a minibatch using stratified proportional sampling.

Divides [0, total_priority] into batch_size equal segments and draws one uniform p from each, then walks the sum-tree to find the leaf whose cumulative-priority interval contains p. Stratification reduces variance vs. drawing batch_size independent uniforms from [0, total] (Schaul §3.3).

Importance-sampling weights are computed as wᵢ = (1 / (N · P(i)))^β and then normalized by max(wⱼ) so the largest weight is exactly 1.0.

Arguments

• batch_size - Number of transitions to draw. Must be > 0.
• beta - IS-weight exponent. Caller anneals from a small value (e.g. 0.4) to 1.0 over training.
• rng - source of randomness.

Panics

Panics if self.is_empty(), batch_size == 0, or the tree’s total priority is zero (all leaves were explicitly set to zero — shouldn’t happen in normal use because push always uses max_priority ≥ ε^α > 0).

pub fn update_priorities(&mut self, indices: &[usize], td_errors: &[f32])

Update the priorities of a set of leaf indices with new TD errors.

For each (idx, td_err) pair, the stored priority becomes (|td_err| + ε)^α. The max_priority tracker is bumped so future Self::push calls inherit at least this value.

Panics

Panics if indices and td_errors have different lengths or if any index is out of range.

pub fn tree(&self) -> &SumTree

Borrow the underlying sum-tree (useful for tests / diagnostics).

Trait Implementations

impl Clone for PrioritizedReplayBuffer

fn clone(&self) -> PrioritizedReplayBuffer

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PrioritizedReplayBuffer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.