Skip to main content

encode

interscript_rs::imf::tokens

Function encode 

Source 
pub fn encode(text: &str) -> Vec<i64>
Expand description

tokens.encode: UTF-8 bytes b -> ids b+3, then one trailing EOS.