pub enum PromptEncoder {
Sequence {
tokenizer: Bpe,
encoder: TextEncoder,
chat: Option<ChatPrompt>,
},
T5 {
tokenizer: Unigram,
encoder: T5Encoder,
length: usize,
},
Clip {
tokenizer: Bpe,
encoder: ClipTextEncoder,
skip: usize,
},
ClipPair {
tokenizer: Bpe,
encoder: ClipTextEncoder,
second_tokenizer: Bpe,
second: ClipTextEncoder,
skip: usize,
},
ClipPairAndT5 {
tokenizer: Bpe,
encoder: ClipTextEncoder,
second_tokenizer: Bpe,
second: ClipTextEncoder,
skip: usize,
third_tokenizer: Unigram,
third: T5Encoder,
length: usize,
},
}Expand description
The encoder a pipeline turns its prompt into hidden states with, holding the tokenizer it was trained with.
Which one a model uses is part of the model: FLUX.2 encodes with a LLaMA-family decoder, FLUX.1 and Stable Diffusion 3 with T5.
Variants§
Sequence
A LLaMA-family decoder — Qwen2, Qwen3, LLaMA, Mistral — over byte-level byte-pair encoding.
Fields
encoder: TextEncoderchat: Option<ChatPrompt>Set for a model that reads its prompt through a chat template and from several layers at once, which is FLUX.2. Left empty, the prompt is read bare and from the last layer.
T5
A T5 encoder over sentencepiece, padded to the length the model was tuned for.
Clip
One CLIP tower, which is how Stable Diffusion 1.x and 2.x read a
prompt. skip is how many layers from the end of the tower to leave
off: zero for 1.x, one for the 2.x line.
ClipPair
Two CLIP towers side by side, which is how Stable Diffusion XL reads one: the per-token states are laid beside each other and the pooled vector comes from the second tower alone.
ClipPairAndT5
Two CLIP towers and a T5 encoder, which is how Stable Diffusion 3 reads a prompt. The CLIP states are laid side by side and padded out to T5’s width, the T5 states follow them in the sequence, and the pooled vector is both towers’ pooled vectors side by side.
Implementations§
Source§impl PromptEncoder
impl PromptEncoder
Sourcepub fn pooled_dim(&self) -> Option<usize>
pub fn pooled_dim(&self) -> Option<usize>
How wide a pooled prompt vector this encoder produces, if it produces one.
T5 does not: the models that use it carry a CLIP tower for this, which
is what ImagePipeline::with_pooled takes.
Auto Trait Implementations§
impl Forget for PromptEncoder
impl Freeze for PromptEncoder
impl Move for PromptEncoder
impl RefUnwindSafe for PromptEncoder
impl Send for PromptEncoder
impl Sync for PromptEncoder
impl Unpin for PromptEncoder
impl UnsafeUnpin for PromptEncoder
impl UnwindSafe for PromptEncoder
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more