Module tokenizer 

lists out the available tokenizers for different OpenAI models.

Enums

Tokenizer

Enum representing the available tokenizers for different OpenAI models.

Functions

get_tokenizer

Returns the tokenizer type used by a model.