Skip to main content

Dit

Struct Dit 

Source
pub struct Dit { /* private fields */ }
Expand description

The denoising transformer.

Implementations§

Source§

impl Dit

Source

pub fn config(&self) -> &DitConfig

The configuration this model was built for.

Source

pub fn pooled_dim(&self) -> Option<usize>

The width of the pooled prompt vector this checkpoint takes, if it takes one at all. Read from the weights rather than the configuration, because it is the weights that have to match.

Source

pub fn set_conditioning(&mut self, conditioning: Conditioning)

Hands the model the prompt it should denoise towards.

Denoiser::denoise reads it, so this has to be called before a sampling run and again for every new prompt.

Source

pub fn set_references( &mut self, references: Option<References>, ) -> Result<(), NetworkError>

Hands it reference images to edit from, or takes them away with None.

Every forward pass then attends over the references’ tokens as well and drops their outputs; only a model with four rotary axes (FLUX.2) has positions to tell them apart.

Source

pub fn set_unconditional(&mut self, conditioning: Conditioning)

Hands it the prompt to push away from, which is the empty one for classifier-free guidance.

A distilled model needs none: it was trained with the guidance already in its weights, and a second pass per step would halve the speed for nothing.

Source

pub fn load( path: impl AsRef<Path>, prefix: &str, config: DitConfig, ) -> Result<Self, NetworkError>

Reads a denoiser out of a checkpoint under the key names FLUX ships with.

path is a .safetensors file or the index of a sharded one, and prefix is what the keys start with — empty for a standalone transformer file, "model.diffusion_model." inside a full pipeline.

Source

pub fn load_at( path: impl AsRef<Path>, prefix: &str, config: DitConfig, precision: Precision, ) -> Result<Self, NetworkError>

The same, holding every projection at whatever precision is asked for.

Precision::Q8 is a quarter of the memory and about a fifth slower per step on this machine, because the byte path multiplies one token at a time where the float path multiplies a block. It is what makes a model that does not fit run at all.

Source

pub fn read( file: &mut ShardedSafeTensors, prefix: &str, config: DitConfig, ) -> Result<Self, NetworkError>

The same, from a checkpoint that is already open.

Both published spellings are read: the one Black Forest Labs ships, and the one diffusers repacks the same weights under. Which is in the file is a question the file answers, so it is not asked of the caller.

Source

pub fn read_at( file: &mut ShardedSafeTensors, prefix: &str, config: DitConfig, precision: Precision, ) -> Result<Self, NetworkError>

The same, at a chosen precision.

Source

pub fn quantize(&mut self)

Stores every projection at one byte per value, a quarter of the memory.

Inference only and one-way: the f32 weights are dropped. The normalization scales and the biases stay as they are, because they are a thousandth of the model and the rounding shows in them.

Source

pub fn forward( &self, patches: &Matrix, timestep: f32, ) -> Result<Matrix, NetworkError>

One forward pass: patches in, predicted velocity out.

patches is [height * width, in_channels], the latent packed into the patch grid the conditioning names.

Source

pub fn forward_with( &self, patches: &Matrix, timestep: f32, conditioning: &Conditioning, ) -> Result<Matrix, NetworkError>

The same pass against a conditioning handed in rather than the one the model is holding, which is how the unconditional half of a guided step runs.

Trait Implementations§

Source§

impl Denoiser for Dit

Source§

fn denoise( &mut self, latents: &Matrix, sigma: f32, ) -> Result<Matrix, NetworkError>

The prediction for latents at noise level sigma. Read more
Source§

fn denoise_unconditional( &mut self, latents: &Matrix, sigma: f32, ) -> Result<Option<Matrix>, NetworkError>

The same prediction with the conditioning dropped, for classifier-free guidance. Read more
Source§

impl From<Dit> for ImageDenoiser

Source§

fn from(denoiser: Dit) -> Self

Converts to this type from the input type.

Auto Trait Implementations§

§

impl Forget for Dit

§

impl Freeze for Dit

§

impl Move for Dit

§

impl RefUnwindSafe for Dit

§

impl Send for Dit

§

impl Sync for Dit

§

impl Unpin for Dit

§

impl UnsafeUnpin for Dit

§

impl UnwindSafe for Dit

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> Pointable for T

Source§

const ALIGN: usize

The alignment of pointer.
Source§

type Init = T

The type for initializers.
Source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V