Skip to main content

tokenize_lowercase

Function tokenize_lowercase 

Source
pub fn tokenize_lowercase(name: &str) -> Vec<String>
Expand description

Extract non-style tokens from a font filename stem.

Tokenizes using CamelCase boundaries, hyphens, underscores, and spaces, then filters out style tokens (Bold, Italic, Regular, etc.). Returns lowercased tokens suitable for family name matching.

§Examples

  • "ArialBold"["arial"]
  • "NotoSansJP-Regular"["noto", "sans", "jp"]
  • "HelveticaNeue-BoldItalic"["helvetica", "neue"] Tokenize a name into lowercase tokens (no style filtering).

Useful for priority scoring where style tokens like “Bold” are still relevant.