Struct Layout 

pub struct Layout {
    pub seq_len: usize,
    pub segments: Vec<Segment>,
    pub pos: Vec<[f64; 3]>,
    pub text_len: usize,
    pub audio_t: usize,
    pub latent_t: usize,
    pub lat_h: usize,
    pub lat_w: usize,
    pub frame_rows: usize,
    pub text_tags: Vec<u8>,
}

The static structure of one shape signature: where each stream sits in the sequence and what 3-D position every row carries.

Fields

seq_len: usize

segments: Vec<Segment>

pos: Vec<[f64; 3]>

[seq_len, 3] — (t, h, w), f64 because the axes are fractional.

text_len: usize

audio_t: usize

latent_t: usize

lat_h: usize

lat_w: usize

frame_rows: usize

Rows per latent frame after the 2×2 patch.

text_tags: Vec<u8>

Per-token modality tag for the text span. A vision block inside the prompt carries the VIDEO tag, not the text one.

Implementations

impl Layout

pub fn t2va( text_len: usize, latent_t: usize, lat_h: usize, lat_w: usize, audio_t: usize, ) -> Self

t2va: [text | audio | video], the target streams last and in that order. Keyframe and reference blocks would slot between the text and the audio; this port does text-to-video only.

pub fn fl2va( text_len: usize, latent_t: usize, lat_h: usize, lat_w: usize, audio_t: usize, frames: &[(usize, usize)], text_tags: &[u8], ) -> Self

fl2va: keyframe condition rows sit between the text and the audio, sharing the TARGET spatial grid, each pinned to the time coordinate of the frame it stands for — the first frame at the text’s end, the last one a whole clip further on, minus one span. They never advance the cursor, so audio and video still start where they would have.

frames gives each keyframe’s pixel index and the clip’s total, and text_tags the per-token modality of the prompt span.

pub fn ref2va( text_len: usize, latent_t: usize, lat_h: usize, lat_w: usize, audio_t: usize, refs: &[Ref], text_tags: &[u8], ) -> Self

ref2va: reference images, audio and clips ahead of the target streams. Unlike a keyframe, a reference ADVANCES the cursor — each block occupies its own stretch of the time axis, and the target audio and video begin after the last of them.

pub fn streaming( text_len: usize, text_tags: &[u8], lat_h: usize, lat_w: usize, ctx_video: &[usize], cur_video: &[usize], ctx_audio: &[usize], cur_audio: &[usize], ) -> Self

The chunk-causal layout: one chunk being denoised, and the chunks it is allowed to see.

RAVEN generates a clip chunk by chunk, each one extrapolated from what came before instead of denoised as one bidirectional clip. Its attention pattern is sink chunks from the start plus a sliding window of recent ones — the reference implements that with a KV cache; the same pattern falls out of simply not packing the rows a chunk may not see, which is what this builds. The cache is then an optimization of this, not a prerequisite.

Positions stay ABSOLUTE. A chunk five steps in must carry the RoPE coordinates it would have had in the whole clip, or the model is told it is generating the opening again.

Rows come out as [text | audio: ctx then current | video: ctx then current]: the audio and video blocks stay contiguous because the patchifier hands them over that way, and audio is channel-major, so its context is two runs — one per channel.

pub fn video_block(&self) -> (usize, usize)

The contiguous span the video rows occupy — context and current together, because the patchifier produces them as one block.

pub fn audio_block(&self) -> (usize, usize)

The same for audio.

pub fn cond_rows(&self) -> usize

How many keyframe condition rows the layout carries.