hf-cache-reader 0.1.0

Read the local Hugging Face cache without network access
Documentation
  • Coverage
  • 100%
    52 out of 52 items documented0 out of 13 items with examples
  • Size
  • Source code size: 35.2 kB This is the summed size of all the files inside the crates.io package for this release.
  • Documentation size: 474.4 kB This is the summed size of all files generated by rustdoc for all configured targets
  • Ø build duration
  • this release: 6s Average build duration of successful builds.
  • all releases: 6s Average build duration of successful builds in releases after 2024-10-23.
  • Links
  • jbg/hf-cache-reader
    0 0 0
  • crates.io
  • Dependencies
  • Versions
  • Owners
  • jbg

hf-cache-reader

hf-cache-reader reads the standard Hugging Face cache layout without making network requests. It is synchronous, read-only, and has no runtime dependencies.

The crate understands repositories, snapshots, refs, nested files, and shared content-addressed blobs. It does not download, modify, prune, or interpret model files.

use hf_cache_reader::{resolve_cache_dir, scan_repo, RepoType};

fn main() -> Result<(), hf_cache_reader::Error> {
    let cache_dir = resolve_cache_dir();
    let scan = scan_repo(&cache_dir, RepoType::Model, "google/gemma-2-2b-it")?;

    if let Some(repo) = scan.repo {
        for revision in repo.revisions {
            println!("{}: {} files", revision.commit_hash, revision.files.len());
        }
    }

    Ok(())
}

To inventory every supported repository type in a cache:

fn main() -> Result<(), hf_cache_reader::Error> {
    let cache = hf_cache_reader::scan_cache(hf_cache_reader::resolve_cache_dir())?;
    println!("{} repositories", cache.repos.len());
    Ok(())
}

Cache location

resolve_cache_dir() uses this precedence:

  1. HF_HUB_CACHE
  2. HUGGINGFACE_HUB_CACHE
  3. $HF_HOME/hub
  4. $XDG_CACHE_HOME/huggingface/hub
  5. $HOME/.cache/huggingface/hub
  6. $USERPROFILE/.cache/huggingface/hub
  7. The platform temporary directory under huggingface/hub

An explicit path passed to scan_cache or scan_repo is preferable when an application already owns cache configuration.

Layout

The reader recognizes the standard layout documented by Hugging Face:

<cache>/
  models--<owner>--<name>/
    refs/<ref>
    snapshots/<commit>/<file>
    blobs/<content>

Dataset, Space, and Kernel repositories use the corresponding datasets, spaces, and kernels prefixes.

Malformed or dangling entries below a readable cache root are reported as ScanWarning values. A missing cache directory is treated as an empty cache.