maolan-generate-0.0.13
- maolan-generate 0.0.13
- Docs.rs crate page
- BSD-2-Clause
- 04 August 2026
- Links
- Homepage
- Repository
- crates.io
- Source
- Owners
- mekanix
- Dependencies
- anyhow ^1.0.104 normal
  burn ^0.21.0 normal
  burn-ndarray ^0.21.0 normal
  burn-store ^0.21.0 normal
  half ^2.7.1 normal
  hf-hub ^0.5.0 normal
  maolan-llama ^0.0.1 normal
  midly ^0.5.3 normal
  oxideav-basic ^0.0.9 normal
  oxideav-core ^0.1.34 normal
  rand ^0.10.2 normal
  rayon ^1.12.0 normal
  safetensors ^0.8.0 normal
  sentencepiece-rust ^0.1.1 normal
  serde ^1.0.229 normal
  serde_json ^1.0.151 normal
  symphonia ^0.5.4 normal
  tokenizers ^0.23.1 normal
  tokie ^0.0.10 normal
  anyhow ^1.0.104 build
- Versions
- 55.1% of the crate is documented
Go to latest version
Platform
- x86_64-unknown-linux-gnu
Feature flags

docs.rs

Rust

Skip to main content

load_tokenizer

maolan_generate0.0.13

maolan_generate

Function load_tokenizer

pub fn load_tokenizer() -> Result<SentencePieceProcessor>