Struct CodecTransformerLayer 

pub struct CodecTransformerLayer<B: Backend> { /* private fields */ }

Codec decoder transformer layer.

Architecture:

x -> RmsNorm -> CodecAttention(ALiBi, QK-norm, causal, sliding_window) -> LayerScale -> + residual -> x'
x' -> RmsNorm -> SwiGLU -> LayerScale -> + residual -> out

Parameters: 8 MHA heads, 1024 dim, head_dim 128, sliding window per block.

Implementations

impl<B: Backend> CodecTransformerLayer<B>

pub fn new( attention_norm: RmsNorm<B>, attention: CodecAttention<B>, attention_scale: LayerScale<B>, ffn_norm: RmsNorm<B>, ffn: SwiGLU<B>, ffn_scale: LayerScale<B>, ) -> Self

Create a codec transformer layer from loaded components.

pub fn from_safetensors( safetensors: &SafeTensors<'_>, prefix: &str, n_heads: usize, head_dim: usize, sliding_window: usize, norm_eps: f64, device: &B::Device, ) -> Result<Self>

Load a codec transformer layer from SafeTensors.

Arguments

• safetensors - SafeTensors data
• prefix - Weight name prefix (e.g., audio_tokenizer.decoder_blocks.0.layers.0)
• n_heads - Number of attention heads
• head_dim - Per-head dimension
• sliding_window - Sliding window size for this layer
• norm_eps - RMSNorm epsilon
• device - Device for tensor allocation

pub fn forward(&self, x: Tensor<B, 3>) -> Tensor<B, 3>

Forward pass.

Arguments

• x - Input tensor [batch, seq, dim]

Returns

Output tensor [batch, seq, dim]