crate-names

Compact, transparent artifacts of every crate name on crates.io, rebuilt daily from the official database dump, for typeahead and default-version lookup. Plus a small sans-io Rust library for reading them.

The artifacts

Published daily to the rolling artifacts release:

All are zstd-compressed TSV, one crate per line, sorted by the crate's folded name — ASCII-lowercased, with - and _ the same character — and guaranteed sorted and (for names and facets) three-fields-per-line by the builder. Because crate names cannot contain tabs or newlines, and description whitespace is flattened to single spaces, there is no escaping: the format is fully processable with standard tools.

Folding is what makes lookups work the way people type: Tokio, tokio and tokio_util all find what you meant. crates.io will not let a new crate take a name that folds onto an existing one, so the folded key is unique registry-wide — the artifacts stay sorted and unique under it, and queries remain two binary searches. Names are stored as spelled; only the ordering is folded.

$ curl -sL https://github.com/jbr/crate-names/releases/download/artifacts/names-v2.tsv.zst \
    | zstd -d | grep '^serde\b'
serde	1.0.228	240

• default_version is crates.io's own notion of the version to present, from the dump's default_versions table.
• rank is the all-time download count quantized to floor(8·log₂(n+1)), saturating at 255. Ordering is meaningful; arithmetic is not.
• keywords and categories are each a space-separated, sorted list — the crate's Cargo.toml keywords (at most five) and crates.io category slugs (nested with ::, e.g. web-programming::http-server). Either field may be empty, but not both: crates declaring neither have no line.
• Freshness: artifacts lag crates.io by up to ~27 hours (daily dump + daily build). Consumers that need same-day version resolution should fall through to the sparse index per crate.

Example

The reader is sans-io: fetch the artifact however you like and hand over the bytes. A sorted list needs no materialized index — construction records line offsets and validates sortedness, and queries are binary searches over the decompressed text.

use crate_names::{CrateNames, Error};

fn typeahead_demo(artifact: &[u8]) -> Result<(), Error> {
    let names = CrateNames::from_zstd(artifact)?;

    // top crates by rank for a prefix
    for entry in names.typeahead("serd", 10) {
        println!("{}\t{}", entry.name, entry.version);
    }

    // name lookup, folded: "Serde" and "serde" are the same crate
    if let Some(serde) = names.get("Serde") {
        println!("serde's default version is {}", serde.version);
    }

    Ok(())
}

The build feature (not needed by consumers) enables build_from_dump, which streams a db-dump.tar.gz and produces the artifacts; the daily workflow is a thin wrapper around it:

$ curl -sL https://static.crates.io/db-dump.tar.gz | crate-names build --out out

Versioning

The wire format and the library are versioned independently: the -v2 in the file names is the format contract, while the crate follows semver as usual.

-v2 re-sorted the artifacts by folded name (see above); a -v1 reader would reject a -v2 artifact as unsorted rather than mis-search it. The -v1 assets are gone — the crate was days old and had no other known consumers.

Safety

This crate uses #![forbid(unsafe_code)].

License