pub fn load_tokenizer(model_path: &Path) -> Result<Tokenizer>
Loads a tokenizer from a model directory or explicit tokenizer.json path.