Skip to main content

Module ltxvae

Module ltxvae 

Source
Expand description

LTX-2.5 convolutional video VAE decoder — the vvae.* half of an ltx-2.5-av container, in Rust with no ML framework underneath.

Latents are [128, F, H, W]; the decoder returns [3, 8·(F−1)+1, 32·H, 32·W] in [-1, 1]. The stack mirrors the encoder, and the checkpoint stores its blocks in the encoder’s order, so decoder_blocks from the config is walked backwards:

un_normalize → conv_in(128→1024)
  res×2 @1024 │ up(2,2,2) 1024→512 │ res×2 @512 │ up(2,2,2) 512→512
  res×4 @512  │ up(2,1,1) 512→256  │ res×6 @256 │ up(1,2,2) 256→128
  res×4 @128
→ PixelNorm → SiLU → conv_out(128→48) → unpatchify(4) → [3, …]

Every convolution is 3×3×3 with zero spatial padding and replicated frames at both ends of the time axis (causal_decoder: false), and every normalization is PixelNorm — RMS over channels at each location, no learned parameters, which is why the checkpoint carries no norm weights. A residual path that changes width normalizes with a 1-group GroupNorm and a 1×1×1 projection; at equal width both are identity, and this decoder is equal-width everywhere.

Convolutions run as im2col + GEMM over the worker pool, in position chunks so the patch buffer stays bounded regardless of resolution.

Structs§

Conv3d
A 3×3×3 convolution with zero spatial padding and replicate padding in time. Weights are [Cout, Cin, 3, 3, 3] flattened to [Cout, Cin·27].
ConvVaeDecoder
The whole decoder, read from a packed ltx-2.5-av container.
DepthToSpaceUp
conv → depth-to-space over (p1, p2, p3) → drop the first frame when the time stride is 2. Channels are laid out (c p1 p2 p3).
ResBlock
PixelNorm → SiLU → conv1 → PixelNorm → SiLU → conv2, plus the residual.
Vol
One [C, F, H, W] volume.

Functions§

unpatchify
[C·q·r, F, H, W][C, F, H·q, W·r] (channels laid out (c r q)).