Module mako::vocab[−][src]

Expand description

Vocab module contains the mako vocab object and the functions to load different vocabularies

Structs

TokenNotFoundError

Custom Error Types

Vocab

Mako Vocab Object

Functions

load_bpe_vocab

Load the BPE vocab from an internal vocab file into a Vocab object

load_wordpiece_vocab

Lod the WordPiece vocab from an internal vocab file into a Vocab object