ferrum-models-0.3.0
- ferrum-models 0.3.0
- Docs.rs crate page
- MIT
- Links
- crates.io
- Source
- Owners
- sizzlecar
- Dependencies
- anyhow ^1.0 normal
  async-trait ^0.1 normal
  bytemuck ^1.23 normal
  candle-core ^0.9.2 normal
  candle-flash-attn ^0.9.2 normal optional
  candle-nn ^0.9.2 normal
  candle-transformers ^0.9.2 normal
  chrono ^0.4 normal
  dirs ^5.0 normal
  ferrum-cuda-kernels ^0.3.0 normal optional
  ferrum-interfaces ^0.3.0 normal
  ferrum-runtime ^0.3.0 normal optional
  ferrum-types ^0.3.0 normal
  futures ^0.3 normal
  futures-util ^0.3 normal
  half ^2.5.0 normal
  hf-hub ^0.4.1 normal
  indicatif ^0.17 normal
  once_cell ^1.19 normal
  parking_lot ^0.12 normal
  pathdiff ^0.2 normal
  rand ^0.8 normal
  reqwest ^0.12 normal
  safetensors ^0.4 normal
  serde ^1.0 normal
  serde_json ^1.0 normal
  thiserror ^1.0 normal
  tokenizers ^0.21 normal
  tokio ^1.35 normal
  toml ^0.8 normal
  tracing ^0.1 normal
  uuid ^1.6 normal
  ferrum-testkit ^0.3.0 dev
  tempfile ^3.8 dev
  tokio-test ^0.4 dev
- Versions
- 45.36% of the crate is documented
Go to latest version
Platform
- x86_64-unknown-linux-gnu
Feature flags

docs.rs

Rust

Skip to main content

Module tokenizer

ferrum_models0.3.0

Module tokenizer

Module Items

Structs

In crate ferrum_models

Module tokenizer

Expand description

Tokenizer 占位实现

Structs§

PlaceholderTokenizer
TokenizerFactory
TokenizerHandle