[−][src]Trait finalfusion::compat::text::ReadTextDims

pub trait ReadTextDims<R> where
    Self: Sized,
    R: BufRead, {
    fn read_text_dims(reader: &mut R) -> Result<Self>;

    fn read_text_dims_lossy(reader: &mut R) -> Result<Self>;
}

Method to construct Embeddings from a text file with dimensions.

This trait defines an extension to Embeddings to read the word embeddings from a text stream. The text must contain as the first line the shape of the embedding matrix:

vocab_size n_components

The remainder of the stream should contain one word embedding per line in the following format:

word0 component_1 component_2 ... component_n

Required methods

fn read_text_dims(reader: &mut R) -> Result<Self>

Read the embeddings from the given buffered reader.

fn read_text_dims_lossy(reader: &mut R) -> Result<Self>

Read the embeddings from the given buffered reader.

In contrast to read_text_dims, this constructor does not fail if a token contains invalid UTF-8. Instead, it will replace invalid UTF-8 characters by the replacement character.

Loading content...

Implementors

impl<R> ReadTextDims<R> for Embeddings<SimpleVocab, NdArray> where
    R: BufRead, [src]

Loading content...