Skip to main content

decode

Function decode 

Source
pub fn decode(token_ids: &[i64]) -> String
Expand description

tokens.decode: stop at EOS; skip pad/unk; (id - 3) mod 256 per byte; reassemble as UTF-8 (lossy, like the reference loader’s errors="replace").