List of all items

Structs

• LocalTokenizer
• accumulator::TextAccumulator
• types::ComputeTokensResult
• types::Content
• types::CountTokensConfig
• types::CountTokensResult
• types::FunctionCall
• types::FunctionDeclaration
• types::FunctionResponse
• types::Part
• types::Schema
• types::TokensInfo
• types::Tool

Enums

• TokenizerError
• types::Contents

Functions

• supported_models
• verify_model_hash

Constants

• MODEL_SHA256
• VOCAB_SIZE