Enum SteKind 

pub enum SteKind {
    Identity,
    ClippedIdentity,
    Tanh,
    HardTanh,
}

Straight-through estimator variants for Op::FakeQuantize’s backward. The forward is the same regardless: discrete clamp(round(x/s)) * s. The choice here affects only the gradient w.r.t. x during training.

• Identity — dx = upstream. The original STE; treats the round as identity in the backward direction. Simplest, fine for moderate bit widths (i4 / i8).

• ClippedIdentity — dx = upstream * (|x| ≤ q_max·s). Zero the gradient when the input was outside the quantization range (i.e. the clamp activated). Stops the optimizer from pushing weights further into saturation.

• Tanh — dx = upstream * (1 - tanh²(x/s)). Smooth surrogate for the round step. Slowly attenuates the gradient as |x| approaches q_max·s. Often best on tight bit widths (i2).

• HardTanh — dx = upstream * (1 - |x/(q_max·s)|).max(0). Piecewise-linear cousin of Tanh; cheaper to compute and nearly as effective.

Variants

Identity

ClippedIdentity

Tanh

HardTanh

Trait Implementations

impl Clone for SteKind

fn clone(&self) -> SteKind

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SteKind

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Default for SteKind

fn default() -> SteKind

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for SteKind

fn deserialize<__D>( __deserializer: __D, ) -> Result<SteKind, <__D as Deserializer<'de>>::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Hash for SteKind

fn hash<__H>(&self, state: &mut __H)

where __H: Hasher,

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for SteKind

fn eq(&self, other: &SteKind) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for SteKind

fn serialize<__S>( &self, __serializer: __S, ) -> Result<<__S as Serializer>::Ok, <__S as Serializer>::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Copy for SteKind

impl Eq for SteKind

impl StructuralPartialEq for SteKind