not_text

Function not_text 

Source
pub fn not_text(token: &str) -> bool
Expand description

Check if a token string is “not text” (is a potential marker boundary).

Returns true if the token is not alphanumeric and not a quote/backslash, OR if it contains CJK characters.