Struct DQNConfig 

pub struct DQNConfig {

    pub learning_rate: f64,
    pub batch_size: usize,
    pub buffer_capacity: usize,
    pub min_buffer_size: usize,
    pub target_update_interval: usize,
    pub gamma: f64,
    pub epsilon_start: f64,
    pub epsilon_end: f64,
    pub epsilon_decay_steps: usize,
    pub max_grad_norm: f64,
    pub soft_update_tau: Option<f64>,
    pub prioritized_replay: bool,
    pub per_alpha: f64,
    pub per_beta_start: f64,
    pub per_beta_end: f64,
    pub per_beta_steps: usize,
    pub per_epsilon: f64,
}

DQN configuration parameters.

Default values target classic CartPole-style discrete control: 50k-capacity replay, 64-sample batches, hard target sync every 500 env steps, linear ε-decay over 10k env steps. These are the same defaults the Stable-Baselines3 DQN baseline uses on CartPole.

Fields

learning_rate: f64

Adam learning rate for the online Q-network.

batch_size: usize

Number of transitions sampled per gradient update.

buffer_capacity: usize

Maximum number of transitions stored in the replay buffer. Older transitions are evicted FIFO once capacity is reached.

min_buffer_size: usize

Minimum number of transitions required before training starts. Until the buffer holds this many transitions the trainer only collects experience (via random or ε-greedy actions) and skips gradient updates.

target_update_interval: usize

Number of environment steps between hard target-net syncs (target ← online).

gamma: f64

Discount factor used in the TD target. With Double-DQN (used unconditionally by crate::train::dqn::DQNTrainerBurn): y = r + γ · (1 - done) · Q_target(s', argmax_a' Q_online(s', a')).

epsilon_start: f64

Initial value of the ε-greedy exploration parameter.

epsilon_end: f64

Final value of ε after the linear decay completes.

epsilon_decay_steps: usize

Number of environment steps over which ε linearly anneals from epsilon_start to epsilon_end. After this many steps ε stays at epsilon_end.

max_grad_norm: f64

Maximum gradient norm for clipping the Q-network update.

soft_update_tau: Option<f64>

Polyak (soft) target update coefficient τ ∈ (0, 1].

When Some(τ), every call to crate::train::dqn::DQNTrainerBurn::maybe_sync_target performs the blend

θ_target ← τ · θ_online + (1 − τ) · θ_target

across every parameter of the target network. This replaces the hard copy that fires every target_update_interval env steps; in soft-update mode target_update_interval is ignored.

When None (default), the trainer falls back to the original hard copy on the interval, preserving byte-for-byte backward compatibility with vanilla DQN.

A typical value is 0.005 (the SB3/Spinning Up default).

prioritized_replay: bool

Use Prioritized Experience Replay (Schaul et al., 2015) in place of the uniform crate::buffer::replay::ReplayBuffer.

When true, the trainer holds a crate::buffer::replay::PrioritizedReplayBuffer and samples transitions proportionally to (|TD error| + ε)^α. Per-sample importance-sampling weights are applied to the Smooth-L1 loss (so high-priority transitions get correspondingly down-weighted updates), and the buffer’s priorities are refreshed after each gradient step with the new TD-error magnitudes.

Defaults to false to preserve the vanilla uniform behavior.

per_alpha: f64

PER priority exponent α ∈ [0, 1]. 0 recovers uniform sampling (purely as a degenerate case — flip Self::prioritized_replay off for the same effect without the sum-tree overhead). 1 is fully proportional to priority. Typical value 0.6 (Schaul §3.2).

per_beta_start: f64

PER importance-sampling exponent at the start of training. Typical value 0.4 (Schaul §3.4).

per_beta_end: f64

PER importance-sampling exponent at the end of the annealing schedule. Always 1.0 in the original paper — full bias correction by the end of training.

per_beta_steps: usize

Number of environment steps over which β linearly anneals from per_beta_start to per_beta_end. After this many steps β stays at per_beta_end. Defaults to epsilon_decay_steps if set to 0 (sentinel “follow the ε schedule”).

per_epsilon: f64

Tiny constant added to |TD error| before raising to α, so transitions with vanishing TD error still have a small chance of being resampled. Typical value 1e-6.

Implementations

impl DQNConfig

pub fn new() -> Self

Create a new default configuration.

pub fn validate(&self) -> Result<()>

Validate configuration parameters.

Returns an Err describing the first invalid field encountered.

pub fn beta_at(&self, env_steps: usize) -> f64

Compute β at a given env-step count under the linear schedule:

β(t) = β_start + (β_end − β_start) · min(t / β_steps, 1)

If per_beta_steps == 0 the trainer falls back to epsilon_decay_steps so callers can leave the field at its default and have β anneal over the same window as ε.

pub fn epsilon_at(&self, env_steps: usize) -> f64

Compute the ε used at a given env-step count under the linear schedule:

ε(t) = max(ε_end, ε_start - (ε_start - ε_end) · t / decay_steps)

pub fn learning_rate(self, lr: f64) -> Self

Set learning rate.

pub fn batch_size(self, size: usize) -> Self

Set minibatch size.

pub fn buffer_capacity(self, capacity: usize) -> Self

Set replay buffer capacity.

pub fn min_buffer_size(self, size: usize) -> Self

Set minimum buffer size before training starts.

pub fn target_update_interval(self, steps: usize) -> Self

Set target update interval (env steps between hard target syncs).

pub fn gamma(self, gamma: f64) -> Self

Set discount factor γ.

pub fn epsilon_start(self, eps: f64) -> Self

Set initial ε for ε-greedy exploration.

pub fn epsilon_end(self, eps: f64) -> Self

Set final ε for ε-greedy exploration.

pub fn epsilon_decay_steps(self, steps: usize) -> Self

Set number of env steps over which ε anneals.

pub fn max_grad_norm(self, norm: f64) -> Self

Set maximum gradient norm.

pub fn soft_update_tau(self, tau: f64) -> Self

Enable Polyak (soft) target updates with coefficient τ.

When set, crate::train::dqn::DQNTrainerBurn::maybe_sync_target performs θ_target ← τ · θ_online + (1 − τ) · θ_target on every call (i.e. every env step in the standard rollout loop) instead of the hard copy gated by target_update_interval.

A typical value is 0.005.

pub fn prioritized_replay(self, enabled: bool) -> Self

Enable or disable Prioritized Experience Replay.

pub fn per_alpha(self, alpha: f64) -> Self

Set the PER priority exponent α.

pub fn per_beta_start(self, beta: f64) -> Self

Set the PER importance-sampling exponent at the start of training.

pub fn per_beta_end(self, beta: f64) -> Self

Set the PER importance-sampling exponent at the end of training.

pub fn per_beta_steps(self, steps: usize) -> Self

Set the number of env steps over which β linearly anneals. Pass 0 to follow Self::epsilon_decay_steps.

pub fn per_epsilon(self, eps: f64) -> Self

Set the PER priority floor ε.

Trait Implementations

impl Clone for DQNConfig

fn clone(&self) -> DQNConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DQNConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for DQNConfig

fn default() -> Self

Returns the “default value” for a type.