Struct Conditioning 

pub struct Conditioning {
    pub video_mask: Vec<f32>,
    pub video_clean: Vec<f32>,
    pub audio_mask: Vec<f32>,
    pub audio_clean: Vec<f32>,
    pub refs: Option<RefTokens>,
}

What is held fixed while the rest is denoised. mask[t] = 0 freezes token t at clean[t] and hands the transformer a timestep of zero for it — which is how one encoded image becomes the first frame of a generated shot, and how a whole encoded clip becomes the picture a soundtrack is written for.

Fields

video_mask: Vec<f32>

video_clean: Vec<f32>

audio_mask: Vec<f32>

audio_clean: Vec<f32>

refs: Option<RefTokens>

Extra clean video tokens carried alongside the clip — reference images, at their own positions on the time axis. They are denoised with the sequence and dropped from the result.

Implementations

impl Conditioning

pub fn video_prefix( geo: &Geometry, clean: &[f32], frames: usize, ) -> Conditioning

Freeze the first frames latent frames at clean (patchified).

pub fn video_all(geo: &Geometry, clean: &[f32]) -> Conditioning

Freeze the whole video stream — the picture is given, the sound is what is being generated.

pub fn with_references(self, refs: RefTokens) -> Conditioning

Carry reference images beside the clip. They are frozen (a guide is given, not generated) and cropped off the result, so the render comes back the size the caller asked for.

pub fn with_audio_all(self, geo: &Geometry, clean: &[f32]) -> Conditioning

Freeze the whole soundtrack — the sound is given, the picture is what is being generated.

Trait Implementations

impl Clone for Conditioning

fn clone(&self) -> Conditioning

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Default for Conditioning

fn default() -> Conditioning

Returns the “default value” for a type.