Skip to main content

code_analyzer

Function code_analyzer 

Source
pub fn code_analyzer() -> TextAnalyzer
Expand description

Build a tantivy TextAnalyzer that tokenizes, expands camelCase/snake_case identifiers into sub-tokens, then lowercases everything.

CodeSplitFilter must run before LowerCaser so that camelCase boundaries (uppercase letters) are still visible when splitting.