Skip to main content

ClipTextEncoder

Struct ClipTextEncoder 

Source
pub struct ClipTextEncoder { /* private fields */ }
Expand description

A CLIP text tower.

Implementations§

Source§

impl ClipTextEncoder

Source

pub fn config(&self) -> &ClipTextConfig

The configuration this tower was built for.

Source

pub fn load( path: impl AsRef<Path>, prefix: &str, config: ClipTextConfig, ) -> Result<Self, NetworkError>

Reads a tower out of a checkpoint.

prefix is what the keys start with, which is empty for the files published as text_encoder/model.safetensors.

Source

pub fn load_at( path: impl AsRef<Path>, prefix: &str, config: ClipTextConfig, precision: Precision, ) -> Result<Self, NetworkError>

The same, holding the projections at a chosen precision.

Source

pub fn read_at( file: &mut ShardedSafeTensors, prefix: &str, config: ClipTextConfig, precision: Precision, ) -> Result<Self, NetworkError>

The same, from a checkpoint that is already open.

Source

pub fn tokenize( &self, tokenizer: &Bpe, prompt: &str, ) -> Result<Vec<u32>, NetworkError>

The token ids for a prompt: the start token, the prompt, the end token, and padding out to the length the position table covers.

Padding is what the published pipelines feed, and the pooled vector is read from the end token rather than the last position because of it. The padding is end tokens unless ClipTextEncoder::set_pad_token said otherwise. Attention is causal, so the padding cannot change the pooled state, but a denoiser cross-attends to every position, padding included, so it has to be the token the model was trained with.

Source

pub fn add_token( &mut self, tokenizer: &mut Bpe, token: &str, vectors: &Matrix, ) -> Result<u32, NetworkError>

Adds a textual inversion: token joins the tokenizer and stands for the rows of vectors, which join the embedding table.

Source

pub fn set_pad_token(&mut self, id: u32)

Pads prompts with this token from now on. Stable Diffusion XL’s second tokenizer pads with ! rather than the end token, and its UNet reads the padded positions.

Source

pub fn forward(&self, ids: &[u32]) -> Result<Matrix, NetworkError>

The hidden states for a sequence of token ids, [tokens, d_model].

Source

pub fn forward_skipping( &self, ids: &[u32], skip: usize, ) -> Result<(Matrix, Matrix), NetworkError>

The same pass, also handing back the states as they were skip layers from the end.

Stable Diffusion XL reads its prompt from the second-to-last layer and its pooled vector from the end of the tower, so both come back here and the tower runs once. skip of zero makes the two the same states.

Source

pub fn pool( &self, ids: &[u32], hidden: &Matrix, ) -> Result<Vec<f32>, NetworkError>

The pooled vector: the hidden state at the end token, through the projection if the checkpoint has one.

Source

pub fn encode( &self, tokenizer: &Bpe, prompt: &str, ) -> Result<(Matrix, Vec<f32>), NetworkError>

Tokenizing, running and pooling in one call.

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> Pointable for T

Source§

const ALIGN: usize

The alignment of pointer.
Source§

type Init = T

The type for initializers.
Source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V