pub struct ClipTextEncoder { /* private fields */ }Expand description
A CLIP text tower.
Implementations§
Source§impl ClipTextEncoder
impl ClipTextEncoder
Sourcepub fn config(&self) -> &ClipTextConfig
pub fn config(&self) -> &ClipTextConfig
The configuration this tower was built for.
Sourcepub fn load(
path: impl AsRef<Path>,
prefix: &str,
config: ClipTextConfig,
) -> Result<Self, NetworkError>
pub fn load( path: impl AsRef<Path>, prefix: &str, config: ClipTextConfig, ) -> Result<Self, NetworkError>
Reads a tower out of a checkpoint.
prefix is what the keys start with, which is empty for the files
published as text_encoder/model.safetensors.
Sourcepub fn load_at(
path: impl AsRef<Path>,
prefix: &str,
config: ClipTextConfig,
precision: Precision,
) -> Result<Self, NetworkError>
pub fn load_at( path: impl AsRef<Path>, prefix: &str, config: ClipTextConfig, precision: Precision, ) -> Result<Self, NetworkError>
The same, holding the projections at a chosen precision.
Sourcepub fn read_at(
file: &mut ShardedSafeTensors,
prefix: &str,
config: ClipTextConfig,
precision: Precision,
) -> Result<Self, NetworkError>
pub fn read_at( file: &mut ShardedSafeTensors, prefix: &str, config: ClipTextConfig, precision: Precision, ) -> Result<Self, NetworkError>
The same, from a checkpoint that is already open.
Sourcepub fn tokenize(
&self,
tokenizer: &Bpe,
prompt: &str,
) -> Result<Vec<u32>, NetworkError>
pub fn tokenize( &self, tokenizer: &Bpe, prompt: &str, ) -> Result<Vec<u32>, NetworkError>
The token ids for a prompt: the start token, the prompt, the end token, and padding out to the length the position table covers.
Padding is what the published pipelines feed, and the pooled vector is
read from the end token rather than the last position because of it.
The padding is end tokens unless ClipTextEncoder::set_pad_token
said otherwise. Attention is causal, so the padding cannot change the
pooled state, but a denoiser cross-attends to every position, padding
included, so it has to be the token the model was trained with.
Sourcepub fn add_token(
&mut self,
tokenizer: &mut Bpe,
token: &str,
vectors: &Matrix,
) -> Result<u32, NetworkError>
pub fn add_token( &mut self, tokenizer: &mut Bpe, token: &str, vectors: &Matrix, ) -> Result<u32, NetworkError>
Adds a textual inversion: token joins the tokenizer and stands for
the rows of vectors, which join the embedding table.
Sourcepub fn set_pad_token(&mut self, id: u32)
pub fn set_pad_token(&mut self, id: u32)
Pads prompts with this token from now on. Stable Diffusion XL’s second
tokenizer pads with ! rather than the end token, and its UNet reads
the padded positions.
Sourcepub fn forward(&self, ids: &[u32]) -> Result<Matrix, NetworkError>
pub fn forward(&self, ids: &[u32]) -> Result<Matrix, NetworkError>
The hidden states for a sequence of token ids, [tokens, d_model].
Sourcepub fn forward_skipping(
&self,
ids: &[u32],
skip: usize,
) -> Result<(Matrix, Matrix), NetworkError>
pub fn forward_skipping( &self, ids: &[u32], skip: usize, ) -> Result<(Matrix, Matrix), NetworkError>
The same pass, also handing back the states as they were skip layers
from the end.
Stable Diffusion XL reads its prompt from the second-to-last layer and
its pooled vector from the end of the tower, so both come back here and
the tower runs once. skip of zero makes the two the same states.
Auto Trait Implementations§
impl Forget for ClipTextEncoder
impl Freeze for ClipTextEncoder
impl Move for ClipTextEncoder
impl RefUnwindSafe for ClipTextEncoder
impl Send for ClipTextEncoder
impl Sync for ClipTextEncoder
impl Unpin for ClipTextEncoder
impl UnsafeUnpin for ClipTextEncoder
impl UnwindSafe for ClipTextEncoder
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more