diffctx-1.14.0
- diffctx 1.14.0
- Permalink
- Docs.rs crate page
- Apache-2.0
- 16 August 2026
- Links
- Homepage
- Repository
- crates.io
- Source
- Owners
- nikolay-e
- Dependencies
- aho-corasick ^1.1 normal
  anyhow ^1 normal
  clap ^4 normal
  glob ^0.3 normal
  libc ^0.2 normal
  mimalloc ^0.1 normal
  once_cell ^1 normal
  pyo3 ^0.29 normal optional
  rayon ^1.10 normal
  regex ^1 normal
  rustc-hash ^2 normal
  serde ^1 normal
  serde_json ^1 normal
  serde_yaml ^0.9 normal
  similar ^3 normal
  thiserror ^2 normal
  tiktoken-rs =0.12.0 normal
  tracing ^0.1 normal
  tracing-subscriber ^0.3 normal
  tree-sitter ^0.25 normal
  tree-sitter-bash ^0.25 normal optional
  tree-sitter-c ^0.24 normal optional
  tree-sitter-c-sharp ^0.23 normal optional
  tree-sitter-clojure ^0.1 normal optional
  tree-sitter-cmake ^0.7 normal optional
  tree-sitter-cpp ^0.23 normal optional
  tree-sitter-css ^0.25 normal optional
  tree-sitter-dart ^0.2 normal optional
  tree-sitter-elixir ^0.3 normal optional
  tree-sitter-erlang ^0.20 normal optional
  tree-sitter-go ^0.25 normal optional
  tree-sitter-graphql ^0.1 normal optional
  tree-sitter-groovy ^0.1 normal optional
  tree-sitter-haskell ^0.23 normal optional
  tree-sitter-hcl ^1.1 normal optional
  tree-sitter-html ^0.23 normal optional
  tree-sitter-java ^0.23 normal optional
  tree-sitter-javascript ^0.25 normal optional
  tree-sitter-json ^0.24 normal optional
  tree-sitter-julia ^0.23 normal optional
  tree-sitter-lua ^0.5 normal optional
  tree-sitter-make ^1.1 normal optional
  tree-sitter-nix ^0.3 normal optional
  tree-sitter-objc ^3.0 normal optional
  tree-sitter-ocaml ^0.25 normal optional
  tree-sitter-php ^0.24 normal optional
  tree-sitter-prisma-io ^1.6 normal optional
  tree-sitter-python ^0.25 normal optional
  tree-sitter-r ^1.2 normal optional
  tree-sitter-ruby ^0.23 normal optional
  tree-sitter-rust ^0.24 normal optional
  tree-sitter-scala ^0.26 normal optional
  tree-sitter-svelte-ng ^1.0 normal optional
  tree-sitter-swift ^0.7 normal optional
  tree-sitter-typescript ^0.23 normal optional
  tree-sitter-yaml ^0.7 normal optional
  tree-sitter-zig ^1.1 normal optional
  wait-timeout ^0.2 normal
  walkdir ^2 normal
  libtest-mimic ^0.8 dev
  tempfile ^3 dev
- Versions
- 12.01% of the crate is documented
Platform
- x86_64-unknown-linux-gnu
Feature flags

docs.rs

Rust

Skip to main content

Module tokenizer

_diffctx1.14.0

Module tokenizer

Module Items

Enums
Functions

In crate _diffctx

Module tokenizer

Enums§

TokenizerError

Functions§

count_tokens
try_count_tokens