Struct QuantizePerTokenArgs 

pub struct QuantizePerTokenArgs<'a, TIn: Element, TOut: IntElement> {
    pub input: TensorRef<'a, TIn, 2>,
    pub scale: TensorRef<'a, TIn, 1>,
    pub zero_point: TensorRef<'a, i32, 1>,
    pub output: TensorMut<'a, TOut, 2>,
}

Args bundle for a quantize_per_token forward launch.

Fields

input: TensorRef<'a, TIn, 2>

Input [N, D] in FP.

scale: TensorRef<'a, TIn, 1>

Per-row scale [N] in FP.

zero_point: TensorRef<'a, i32, 1>

Per-row zero-point [N] in i32.

output: TensorMut<'a, TOut, 2>

Output [N, D] in int.