Struct AdaptiveQuantizer 

pub struct AdaptiveQuantizer { /* private fields */ }

Adaptive quantizer that adjusts step size based on local signal statistics

Uses a sliding window to compute local variance and adapts quantization step size accordingly. High-variance regions get finer quantization.

Implementations

impl AdaptiveQuantizer

pub fn new( bits: u8, window_size: usize, adaptation_strength: f32, global_min: f32, global_max: f32, ) -> TokenizerResult<Self>

Create a new adaptive quantizer

pub fn quantize_adaptive( &self, signal: &Array1<f32>, ) -> TokenizerResult<Array1<i32>>

Quantize entire signal with adaptation

Trait Implementations

impl Clone for AdaptiveQuantizer

fn clone(&self) -> AdaptiveQuantizer

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for AdaptiveQuantizer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Quantizer for AdaptiveQuantizer

fn quantize(&self, value: f32) -> i32

Quantize a continuous value to discrete level

fn dequantize(&self, level: i32) -> f32

Dequantize a discrete level back to continuous value

fn num_levels(&self) -> usize

Get number of quantization levels

impl SignalTokenizer for AdaptiveQuantizer

fn encode(&self, signal: &Array1<f32>) -> TokenizerResult<Array1<f32>>

Encode a continuous signal into tokens/embeddings

fn decode(&self, tokens: &Array1<f32>) -> TokenizerResult<Array1<f32>>

Decode tokens/embeddings back to continuous signal

fn embed_dim(&self) -> usize

Get the embedding dimension

fn vocab_size(&self) -> usize

Get the vocabulary size (for discrete tokenizers, 0 for continuous)