Struct LinearQFunction 

pub struct LinearQFunction<T: Float + Debug + Send + Sync + 'static> { /* private fields */ }

Action-value function linear in the state-action features

φ(s,a) = [ s ; a ; vec(s aᵀ) ]        Q(s,a) = w · φ(s,a)

The bilinear s aᵀ block is what makes the deterministic policy gradient non-trivial: without it ∇_a Q would be a constant vector and every DPG step would push the actor to the same saturated action regardless of the state.

ValueNetwork::evaluate_value deliberately returns OptimError::UnsupportedOperation: a Q function has no state value without a policy to average over, and silently returning Q(s, 0) would be a trap.

Implementations

impl<T: Float + Debug + Send + Sync + 'static> LinearQFunction<T>

pub fn new(state_dim: usize, action_dim: usize) -> Result<Self>

Create a zero-initialized Q function.

pub fn state_dim(&self) -> usize

State dimension.

pub fn action_dim(&self) -> usize

Action dimension.

pub fn weights(&self) -> &Array1<T>

Current weights over the state-action feature vector.

Trait Implementations

impl<T: Clone + Float + Debug + Send + Sync + 'static> Clone for LinearQFunction<T>

fn clone(&self) -> LinearQFunction<T>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<T: Debug + Float + Debug + Send + Sync + 'static> Debug for LinearQFunction<T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<T: Float + Debug + Send + Sync + 'static> QNetwork<T> for LinearQFunction<T>

fn evaluate_q( &self, states: &Array2<T>, actions: &Array2<T>, ) -> Result<Array1<T>>

Evaluate Q(s, a) for a batch of state-action pairs.

fn q_gradient( &self, states: &Array2<T>, actions: &Array2<T>, residuals: &Array1<T>, ) -> Result<HashMap<String, Array1<T>>>

Gradient of a residual-weighted sum of Q predictions: ∂/∂θ Σᵢ rᵢ · Q(sᵢ, aᵢ).

fn action_gradient( &self, states: &Array2<T>, actions: &Array2<T>, ) -> Result<Array2<T>>

∇_a Q(s, a) for each row, shape (n_samples, action_dim).

impl<T: Float + Debug + Send + Sync + 'static> ValueNetwork<T> for LinearQFunction<T>

fn evaluate_value(&self, observations: &Array2<T>) -> Result<Array1<T>>

Evaluate value function for given observations

fn update_parameters( &mut self, deltas: &HashMap<String, Array1<T>>, ) -> Result<()>

Add a parameter delta to the value-function parameters.

fn get_parameters(&self) -> HashMap<String, Array1<T>>

Get current value function parameters

fn value_gradient( &self, observations: &Array2<T>, residuals: &Array1<T>, ) -> Result<HashMap<String, Array1<T>>>

Gradient of a residual-weighted sum of value predictions: ∂/∂θ Σᵢ rᵢ · V(sᵢ).