pub fn set_tokenize_metrics_enabled(enabled: bool)
Enable or disable tokenize metrics collection on the hot path.