Skip to main content

get_encoding

tiktoken_wasm

Function get_encoding 

Source 
pub fn get_encoding(name: &str) -> Result<Encoding, JsError>
Expand description

Get an encoding by name.

Supported encodings:

  • "cl100k_base" — GPT-4, GPT-3.5-turbo
  • "o200k_base" — GPT-4o, GPT-4.1, o1, o3
  • "o200k_harmony" — gpt-oss (harmony chat format)
  • "p50k_base" — text-davinci-002/003
  • "p50k_edit" — text-davinci-edit
  • "r50k_base" — GPT-3 (davinci, curie, etc.)
  • "gpt2" — GPT-2 (alias for r50k_base)
  • "llama3" — Meta Llama 3/4
  • "deepseek_v3" — DeepSeek V3/R1
  • "qwen2" — Qwen 2/2.5/3
  • "mistral_v3" — Mistral/Codestral/Pixtral

Throws Error for unknown encoding names.