autotokenizer-0.1.2
- autotokenizer 0.1.2
- Permalink
- Docs.rs crate page
- MIT
- Links
- crates.io
- Source
- Owners
- darkautism
- Dependencies
- hf-hub ^0.4.2 normal
  minijinja ^2.10.0 normal
  serde ^1.0.219 normal
  serde_json ^1.0.140 normal
- Versions
- 0% of the crate is documented
Platform
- x86_64-unknown-linux-gnu
Feature flags

docs.rs

Rust

Crate autotokenizer

autotokenizer0.1.2

All Items

Crate Items

Structs
Enums

Crate autotokenizer

Structs§

AutoTokenizer
DefaultPromptMessage
TokenObj

Enums§

Token