Skip to main content

baseRT_decode_token_raw

Function baseRT_decode_token_raw 

Source
pub unsafe extern "C" fn baseRT_decode_token_raw(
    model: baseRT_model_t,
    token_id: u32,
    out: *mut c_char,
    max_bytes: c_int,
) -> c_int
Expand description

Length-preserving stateless decode (byte-level tokens can contain NUL); returns the true byte length, which may exceed max_bytes.