Skip to main content

PromptEncoder

Enum PromptEncoder 

Source
pub enum PromptEncoder {
    Sequence {
        tokenizer: Bpe,
        encoder: TextEncoder,
        chat: Option<ChatPrompt>,
    },
    T5 {
        tokenizer: Unigram,
        encoder: T5Encoder,
        length: usize,
    },
    Clip {
        tokenizer: Bpe,
        encoder: ClipTextEncoder,
        skip: usize,
    },
    ClipPair {
        tokenizer: Bpe,
        encoder: ClipTextEncoder,
        second_tokenizer: Bpe,
        second: ClipTextEncoder,
        skip: usize,
    },
    ClipPairAndT5 {
        tokenizer: Bpe,
        encoder: ClipTextEncoder,
        second_tokenizer: Bpe,
        second: ClipTextEncoder,
        skip: usize,
        third_tokenizer: Unigram,
        third: T5Encoder,
        length: usize,
    },
}
Expand description

The encoder a pipeline turns its prompt into hidden states with, holding the tokenizer it was trained with.

Which one a model uses is part of the model: FLUX.2 encodes with a LLaMA-family decoder, FLUX.1 and Stable Diffusion 3 with T5.

Variants§

§

Sequence

A LLaMA-family decoder — Qwen2, Qwen3, LLaMA, Mistral — over byte-level byte-pair encoding.

Fields

§tokenizer: Bpe
§encoder: TextEncoder
§chat: Option<ChatPrompt>

Set for a model that reads its prompt through a chat template and from several layers at once, which is FLUX.2. Left empty, the prompt is read bare and from the last layer.

§

T5

A T5 encoder over sentencepiece, padded to the length the model was tuned for.

Fields

§tokenizer: Unigram
§encoder: T5Encoder
§length: usize
§

Clip

One CLIP tower, which is how Stable Diffusion 1.x and 2.x read a prompt. skip is how many layers from the end of the tower to leave off: zero for 1.x, one for the 2.x line.

Fields

§tokenizer: Bpe
§skip: usize
§

ClipPair

Two CLIP towers side by side, which is how Stable Diffusion XL reads one: the per-token states are laid beside each other and the pooled vector comes from the second tower alone.

Fields

§tokenizer: Bpe
§second_tokenizer: Bpe
§skip: usize
§

ClipPairAndT5

Two CLIP towers and a T5 encoder, which is how Stable Diffusion 3 reads a prompt. The CLIP states are laid side by side and padded out to T5’s width, the T5 states follow them in the sequence, and the pooled vector is both towers’ pooled vectors side by side.

Fields

§tokenizer: Bpe
§second_tokenizer: Bpe
§skip: usize
§third_tokenizer: Unigram
§length: usize

Implementations§

Source§

impl PromptEncoder

Source

pub fn d_model(&self) -> usize

How wide one hidden state is.

Source

pub fn pooled_dim(&self) -> Option<usize>

How wide a pooled prompt vector this encoder produces, if it produces one.

T5 does not: the models that use it carry a CLIP tower for this, which is what ImagePipeline::with_pooled takes.

Source

pub fn encode( &self, prompt: &str, ) -> Result<(Matrix, Option<Vec<f32>>), NetworkError>

The hidden states for a prompt, and the pooled vector where the encoder produces one.

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> Pointable for T

Source§

const ALIGN: usize

The alignment of pointer.
Source§

type Init = T

The type for initializers.
Source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V