pub struct AdvancedQuantizationConfig {
pub method: QuantizationMethod,
pub block_size: usize,
pub double_quantization: bool,
pub compute_dtype: ComputeDataType,
pub outlier_threshold: f32,
pub nested_quantization: bool,
}Expand description
Advanced quantization configuration
Fields§
§method: QuantizationMethodQuantization method
block_size: usizeBlock size for block-wise quantization (default: 64)
double_quantization: boolWhether to use double quantization (quantize scaling factors)
compute_dtype: ComputeDataTypeCompute data type for quantization operations
outlier_threshold: f32Outlier threshold for mixed precision (default: 6.0)
nested_quantization: boolWhether to use nested quantization for very large models
Trait Implementations§
Source§impl Clone for AdvancedQuantizationConfig
impl Clone for AdvancedQuantizationConfig
Source§impl Debug for AdvancedQuantizationConfig
impl Debug for AdvancedQuantizationConfig
Auto Trait Implementations§
impl Freeze for AdvancedQuantizationConfig
impl RefUnwindSafe for AdvancedQuantizationConfig
impl Send for AdvancedQuantizationConfig
impl Sync for AdvancedQuantizationConfig
impl Unpin for AdvancedQuantizationConfig
impl UnsafeUnpin for AdvancedQuantizationConfig
impl UnwindSafe for AdvancedQuantizationConfig
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§fn in_current_span(self) -> Instrumented<Self> ⓘ
fn in_current_span(self) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more