Trait QNetwork 

pub trait QNetwork<T: Float + Debug + Send + Sync + 'static>: ValueNetwork<T> {
    // Required method
    fn evaluate_q(
        &self,
        states: &Array2<T>,
        actions: &Array2<T>,
    ) -> Result<Array1<T>>;

    // Provided methods
    fn q_gradient(
        &self,
        states: &Array2<T>,
        actions: &Array2<T>,
        residuals: &Array1<T>,
    ) -> Result<HashMap<String, Array1<T>>> { ... }
    fn action_gradient(
        &self,
        states: &Array2<T>,
        actions: &Array2<T>,
    ) -> Result<Array2<T>> { ... }
}

Action-value (Q) network interface.

The off-policy actor-critic methods (SAC, TD3, DDPG) are built on Q(s, a), not on a state value V(s): without the action argument the deterministic policy gradient ∇_a Q(s, a) does not exist and the critic cannot distinguish the actions it is supposed to rank.

A pure Q network has no intrinsic state value, so it is free to return OptimError::UnsupportedOperation from ValueNetwork::evaluate_value — see linear_models::LinearQFunction.

Required Methods

fn evaluate_q( &self, states: &Array2<T>, actions: &Array2<T>, ) -> Result<Array1<T>>

Evaluate Q(s, a) for a batch of state-action pairs.

Provided Methods

fn q_gradient( &self, states: &Array2<T>, actions: &Array2<T>, residuals: &Array1<T>, ) -> Result<HashMap<String, Array1<T>>>

Gradient of a residual-weighted sum of Q predictions: ∂/∂θ Σᵢ rᵢ · Q(sᵢ, aᵢ).

fn action_gradient( &self, states: &Array2<T>, actions: &Array2<T>, ) -> Result<Array2<T>>

∇_a Q(s, a) for each row, shape (n_samples, action_dim).

This is the term the deterministic policy gradient chains with PolicyNetwork::mean_action_gradient.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl<T: Float + Debug + Send + Sync + 'static> QNetwork<T> for LinearQFunction<T>