plugmem-core-0.1.1
- plugmem-core 0.1.1
- Docs.rs crate page
- MIT
- 29 July 2026
- Links
- Homepage
- Repository
- crates.io
- Source
- Owners
- m62624
- Dependencies
- hashbrown ^0.17.1 normal
  libm ^0.2.16 normal
  plugmem-arena ^0.1.1 normal
  serde ^1.0.229 normal optional
  thiserror ^2.0.18 normal
  unicode-normalization ^0.1.25 normal
  unicode-segmentation ^1.13.3 normal
  xxhash-rust ^0.8.17 normal
  criterion ^0.8.2 dev
  insta ^1.48.0 dev
  proptest ^1.11.0 dev
  serde_json ^1.0.150 dev
- Versions
- 100% of the crate is documented
Go to latest version
Platform
- x86_64-unknown-linux-gnu
Feature flags

docs.rs

Rust

Skip to main content

MAX_TOKEN_BYTES

plugmem_core0.1.1

In plugmem_core::tokenizer

plugmem_core::tokenizer

Constant MAX_TOKEN_BYTES

pub const MAX_TOKEN_BYTES: usize = 64;

Expand description

Upper bound on an emitted token, in bytes.