riptoken 0.3.0

Fast BPE tokenizer for LLMs — a faster, drop-in compatible reimplementation of tiktoken
Documentation

Build #3794269 2026-07-08 08:25:56

# rustc version rustc 1.99.0-nightly (f10db292a 2026-07-07)

# docs.rs version docsrs 0.0.0 (964c90a45859fd8f9076e3cbe58470a5a4f4d4f7 2026-07-01 )

# build log [INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "DOCS_RS=1" "-w" "/opt/rustwide/workdir" "--user" "1001:1001" "4db6b89a62a2a774c88583c2d5a720fcc9ee20ccb35626a37ba831338c63ad47" "/opt/rustwide/cargo-home/bin/cargo" "+nightly" "rustdoc" "--lib" "-Zrustdoc-map" "--config" "build.rustdocflags=[\"--cfg\", \"docsrs\", \"-Z\", \"unstable-options\", \"--emit=invocation-specific\", \"--resource-suffix\", \"-20260707-1.99.0-nightly-f10db292a\", \"--static-root-path\", \"/-/rustdoc.static/\", \"--cap-lints\", \"warn\", \"--extern-html-root-takes-precedence\"]" "--offline" "-Zunstable-options" "--config=doc.extern-map.registries.crates-io=\"https://docs.rs/{pkg_name}/{version}/x86_64-unknown-linux-gnu\"" "-Zrustdoc-scrape-examples" "-j6" "--target" "x86_64-unknown-linux-gnu", kill_on_drop: false }` [INFO] [stderr] warning: Rustdoc did not scrape the following examples because they require dev-dependencies: dfa_spike, parallel_bench [INFO] [stderr] If you want Rustdoc to scrape these examples, then add `doc-scrape-examples = true` [INFO] [stderr] to the [[example]] target configuration of at least one example. [INFO] [stderr] warning: target filter specified, but no targets matched; this is a no-op [INFO] [stderr] Documenting riptoken v0.3.0 (/opt/rustwide/workdir) [INFO] [stderr] Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.82s [INFO] [stderr] Generated /opt/rustwide/target/x86_64-unknown-linux-gnu/doc/riptoken/index.html [INFO] running `Command { std: "docker" "inspect" "4db6b89a62a2a774c88583c2d5a720fcc9ee20ccb35626a37ba831338c63ad47", kill_on_drop: false }`