llm-base-0.1.0-rc2
- llm-base 0.1.0-rc2
- Docs.rs crate page
- MIT OR Apache-2.0
- Links
- Repository
- crates.io
- Source
- Owners
- setzer22
- philpax
- Dependencies
- bytemuck ^1.13.1 normal
  
  ggml ^0.1.0-rc2 normal
  
  half ^2.2.1 normal
  
  memmap2 ^0.5.10 normal
  
  partial_sort ^0.2.0 normal
  
  rand ^0.8.5 normal
  
  serde ^1.0 normal
  
  serde_bytes ^0.11 normal
  
  thiserror ^1.0 normal
- Versions
- 100% of the crate is documented
Go to latest stable release
Platform
- i686-unknown-linux-gnu
- x86_64-unknown-linux-gnu
Feature flags

docs.rs

Rust

TokenId

In llm_base

Type Definition llm_base::TokenId

pub type TokenId = i32;

Expand description

The identifier of a token in a vocabulary.