matrixcode-core-0.4.32
- matrixcode-core 0.4.32
- Docs.rs crate page
- MIT
- Links
- Repository
- crates.io
- Source
- Owners
- bigfish1913
- Dependencies
- anyhow ^1 normal
  async-trait ^0.1 normal
  chrono ^0.4 normal
  clap ^4 normal
  dirs ^5 normal
  dotenvy ^0.15 normal
  env_logger ^0.11 normal
  futures-util ^0.3 normal
  glob ^0.3 normal
  log ^0.4 normal
  lsp-types ^0.95 normal
  notify ^8.0 normal
  once_cell ^1 normal
  regex ^1 normal
  reqwest ^0.13 normal
  rusqlite ^0.32 normal
  serde ^1 normal
  serde_json ^1 normal
  serde_yaml ^0.9 normal
  shell-words ^1 normal
  tiktoken-rs ^0.5 normal
  tokio ^1 normal
  toml ^0.8 normal
  tracing ^0.1 normal
  urlencoding ^2 normal
  uuid ^1 normal
  mockito ^1 dev
  tempfile ^3 dev
  tokio-test ^0.4 dev
- Versions
- 69.2% of the crate is documented
Go to latest version
Platform
- x86_64-unknown-linux-gnu
Feature flags

docs.rs

Rust

Skip to main content

count_tokens

matrixcode_core0.4.32

In matrixcode_core::tokenizer

matrixcode_core::tokenizer

Function count_tokens

pub fn count_tokens(text: &str) -> u32

Expand description

Count tokens in a text string.