Skip to main content

normalize_charset

Function normalize_charset 

Source
pub fn normalize_charset(charset: &str) -> &str
Expand description

Strip line terminators, and nothing else.

The charset’s first character really is U+0020 — a space is one of the classes the model emits. A bare .trim() eats it, which drops the charset from 276 characters to 275 and shifts every index in the decode by one, so every character comes back as its neighbour.