pub struct Dit { /* private fields */ }Expand description
The denoising transformer.
Implementations§
Source§impl Dit
impl Dit
Sourcepub fn pooled_dim(&self) -> Option<usize>
pub fn pooled_dim(&self) -> Option<usize>
The width of the pooled prompt vector this checkpoint takes, if it takes one at all. Read from the weights rather than the configuration, because it is the weights that have to match.
Sourcepub fn set_conditioning(&mut self, conditioning: Conditioning)
pub fn set_conditioning(&mut self, conditioning: Conditioning)
Hands the model the prompt it should denoise towards.
Denoiser::denoise reads it, so this has to be called before a
sampling run and again for every new prompt.
Sourcepub fn set_references(
&mut self,
references: Option<References>,
) -> Result<(), NetworkError>
pub fn set_references( &mut self, references: Option<References>, ) -> Result<(), NetworkError>
Hands it reference images to edit from, or takes them away with None.
Every forward pass then attends over the references’ tokens as well and drops their outputs; only a model with four rotary axes (FLUX.2) has positions to tell them apart.
Sourcepub fn set_unconditional(&mut self, conditioning: Conditioning)
pub fn set_unconditional(&mut self, conditioning: Conditioning)
Hands it the prompt to push away from, which is the empty one for classifier-free guidance.
A distilled model needs none: it was trained with the guidance already in its weights, and a second pass per step would halve the speed for nothing.
Sourcepub fn load(
path: impl AsRef<Path>,
prefix: &str,
config: DitConfig,
) -> Result<Self, NetworkError>
pub fn load( path: impl AsRef<Path>, prefix: &str, config: DitConfig, ) -> Result<Self, NetworkError>
Reads a denoiser out of a checkpoint under the key names FLUX ships with.
path is a .safetensors file or the index of a sharded one, and
prefix is what the keys start with — empty for a standalone
transformer file, "model.diffusion_model." inside a full pipeline.
Sourcepub fn load_at(
path: impl AsRef<Path>,
prefix: &str,
config: DitConfig,
precision: Precision,
) -> Result<Self, NetworkError>
pub fn load_at( path: impl AsRef<Path>, prefix: &str, config: DitConfig, precision: Precision, ) -> Result<Self, NetworkError>
The same, holding every projection at whatever precision is asked for.
Precision::Q8 is a quarter of the memory and about a fifth slower
per step on this machine, because the byte path multiplies one token at
a time where the float path multiplies a block. It is what makes a
model that does not fit run at all.
Sourcepub fn read(
file: &mut ShardedSafeTensors,
prefix: &str,
config: DitConfig,
) -> Result<Self, NetworkError>
pub fn read( file: &mut ShardedSafeTensors, prefix: &str, config: DitConfig, ) -> Result<Self, NetworkError>
The same, from a checkpoint that is already open.
Both published spellings are read: the one Black Forest Labs ships, and the one diffusers repacks the same weights under. Which is in the file is a question the file answers, so it is not asked of the caller.
Sourcepub fn read_at(
file: &mut ShardedSafeTensors,
prefix: &str,
config: DitConfig,
precision: Precision,
) -> Result<Self, NetworkError>
pub fn read_at( file: &mut ShardedSafeTensors, prefix: &str, config: DitConfig, precision: Precision, ) -> Result<Self, NetworkError>
The same, at a chosen precision.
Sourcepub fn quantize(&mut self)
pub fn quantize(&mut self)
Stores every projection at one byte per value, a quarter of the memory.
Inference only and one-way: the f32 weights are dropped. The
normalization scales and the biases stay as they are, because they are
a thousandth of the model and the rounding shows in them.
Sourcepub fn forward(
&self,
patches: &Matrix,
timestep: f32,
) -> Result<Matrix, NetworkError>
pub fn forward( &self, patches: &Matrix, timestep: f32, ) -> Result<Matrix, NetworkError>
One forward pass: patches in, predicted velocity out.
patches is [height * width, in_channels], the latent packed into the
patch grid the conditioning names.
Sourcepub fn forward_with(
&self,
patches: &Matrix,
timestep: f32,
conditioning: &Conditioning,
) -> Result<Matrix, NetworkError>
pub fn forward_with( &self, patches: &Matrix, timestep: f32, conditioning: &Conditioning, ) -> Result<Matrix, NetworkError>
The same pass against a conditioning handed in rather than the one the model is holding, which is how the unconditional half of a guided step runs.
Trait Implementations§
Source§impl Denoiser for Dit
impl Denoiser for Dit
Auto Trait Implementations§
impl Forget for Dit
impl Freeze for Dit
impl Move for Dit
impl RefUnwindSafe for Dit
impl Send for Dit
impl Sync for Dit
impl Unpin for Dit
impl UnsafeUnpin for Dit
impl UnwindSafe for Dit
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more