Skip to main content

tokenize

graphify_serve::search

Function tokenize 

Source 
pub fn tokenize(input: &str) -> Vec<String>
Expand description

Split input on camelCase boundaries, _, ., ::, /, \, -, and whitespace. Returns all-lowercase, non-empty tokens.

ยงExamples (implicit, tested below)

  • "camelCase" -> ["camel", "case"]
  • "foo_bar.baz" -> ["foo", "bar", "baz"]
  • "std::collections::HashMap" -> ["std", "collections", "hash", "map"]
  • "src/main/mod.rs" -> ["src", "main", "mod", "rs"]