Crate hf_fetch_model

Expand description

§hf-fetch-model

Fast HuggingFace model downloads for Rust.

An embeddable library for downloading HuggingFace model repositories with maximum throughput. Wraps hf_hub and adds repo-level orchestration.

§Quick Start

let outcome = hf_fetch_model::download("julien-c/dummy-unknown".to_owned()).await?;
println!("Model at: {}", outcome.inner().display());

§Configured Download

use hf_fetch_model::FetchConfig;

let config = FetchConfig::builder()
    .filter("*.safetensors")
    .filter("*.json")
    .on_progress(|e| {
        println!("{}: {:.1}%", e.filename, e.percent);
    })
    .build()?;

let outcome = hf_fetch_model::download_with_config(
    "google/gemma-2-2b".to_owned(),
    &config,
).await?;
// outcome.is_cached() tells you if it came from local cache
let path = outcome.into_inner();

§Inspect Before Downloading

Read tensor metadata from .safetensors headers via HTTP Range requests — no weight data downloaded. Sharded repos (those with model.safetensors.index.json) work transparently — inspect::inspect_repo_safetensors reads every shard’s header in parallel and returns a flat per-file result list. See examples/candle_inspect.rs for a runnable example, or the Inspect tutorial for a narrative walkthrough.

let results = hf_fetch_model::inspect::inspect_repo_safetensors(
    "EleutherAI/pythia-1.4b", None, None,
).await?;

for (filename, header, _source) in &results {
    println!("{filename}: {} tensors", header.tensors.len());
}

The CLI also exposes hf-fm inspect <repo> [FILE] --check-gpu [N] (v0.10.1) to print a one-line GPU-fit verdict against device N (default 0) using the hypomnesis crate (NVML on Linux/Windows, DXGI on Windows). Adding --context N (v0.10.4) folds in the KV cache at a context length and reports a real fit against weights + KV instead of weights alone — the difference between “fits” and “out-of-memory at token 8000” on a consumer card. The architecture parameters come from the model’s config.json, parsed by the library API that v0.10.4 exposes for downstream reuse: inspect::ModelConfig plus inspect::fetch_model_config / inspect::fetch_model_config_cached (cache-first or cache-only) and the inspect::torch_dtype_bytes helper. The KV math itself — GQA, sliding-window, MLA-skip, and hybrid Mamba/attention layer counting — and the verdict rendering stay binary-only; depend on hypomnesis directly for the raw device-info numbers.

§Cached-file Inspection

Beyond the remote-or-cached .safetensors path above, inspect::inspect_gguf_cached (v0.10.2), inspect::inspect_npz_cached, and inspect::inspect_pth_cached (both v0.10.3) extend inspect to GGUF / NumPy .npz / PyTorch .pth files in the local cache via the anamnesis parser crate. All four formats return the same format-agnostic inspect::SafetensorsHeaderInfo shape, so downstream pipeline steps (filter, tree, dtypes aggregation) work uniformly across formats.

For cached .safetensors files, v0.10.3 also surfaces quantization detection. When inspect::inspect_safetensors_local sees a quantized header (FP8 variants, GPTQ, AWQ, BnB-NF4, BnB-INT8), it populates the new inspect::QuantInfo field with the scheme name and both stored + dequantised byte sizes. Unquantized safetensors and non-safetensors formats leave quant_info as None.

use hf_fetch_model::inspect;
use std::path::Path;

let header = inspect::inspect_safetensors_local(
    Path::new("/path/to/cached/file.safetensors"),
)?;
if let Some(q) = &header.quant_info {
    println!(
        "Quantized as {}: {} stored -> {} dequantised",
        q.scheme, q.stored_bytes, q.dequantized_bytes,
    );
}

Remote inspect for GGUF / NPZ / PTH (via HTTP Range, without going through the cache) is planned for v0.11; until then those formats error early with a “pass –cached after downloading” recovery hint.

For discovery — “what tensor files does this cached repo hold?” — inspect::list_cached_tensor_files (v0.10.5) enumerates (filename, size) pairs across all four formats without parsing any headers, with inspect::is_supported_tensor_file / inspect::SUPPORTED_TENSOR_EXTENSIONS as the shared extension predicate. The .safetensors-only inspect::list_cached_safetensors (v0.9.7) remains for callers that want exactly that subset. These back the CLI’s inspect --list, numeric-index, and --pick flows.

§`HuggingFace` Cache

Downloaded files are stored in the standard HuggingFace cache directory (~/.cache/huggingface/hub/), ensuring compatibility with Python tooling.

§Cache Management

v0.10.0 adds library APIs for inspecting, verifying, and pruning the local cache. cache::cache_summary enumerates every cached repo with size and file counts; cache::repo_status gives a per-file Complete / Partial / Missing / Excluded breakdown for one repo (since v0.10.5, partials are attributed per-file via each file’s own blobs/<sha256>.chunked.part temp blob rather than a repo-level heuristic); cache::verify_cache re-checks SHA256 digests of cached files against HuggingFace LFS metadata; and cache::find_partial_files locates .chunked.part orphans from interrupted downloads.

For long verifications (multi-GiB safetensors files), drive cache::verify_cache_with_progress with an Fn callback that receives cache::VerifyEvents so a CLI or GUI can render a spinner or progress bar without polling.

use hf_fetch_model::cache::{self, VerifyStatus};

let results = cache::verify_cache("google/gemma-2-2b-it", None, None).await?;
let ok = results
    .iter()
    .filter(|r| matches!(r.status, VerifyStatus::Ok))
    .count();
let mismatch = results
    .iter()
    .filter(|r| matches!(r.status, VerifyStatus::Mismatch { .. }))
    .count();
println!("{}/{} files verified, {} mismatches", ok, results.len(), mismatch);

§Download Durability

Multi-connection downloads survive interruption. When a download is aborted by FetchConfigBuilder::timeout_per_file (default 300 s), Ctrl-C, panic, or a transient chunk error, the partial .chunked.part file plus a small per-chunk progress sidecar are kept on disk. The next call to download_with_config for the same file picks up where it stopped — each parallel chunk sends a fresh Range request that skips the bytes it already has — provided the upstream etag still matches. On etag change, schema-version mismatch, or a different FetchConfigBuilder::connections_per_file count, the partial is discarded and a fresh download starts.

For slow connections on multi-GiB files, raise the per-file budget to match real throughput:

use std::time::Duration;
use hf_fetch_model::FetchConfig;

let config = FetchConfig::builder()
    .timeout_per_file(Duration::from_secs(1800))
    .build()?;

§Authentication

Set the HF_TOKEN environment variable to access private or gated models, or use FetchConfig::builder().token().

Gated repos (Meta Llama, Google Gemma, …) additionally require accepting the license on the model’s HuggingFace page — once per gated family (a Llama 3.2 grant does not cover Llama 3.1). download() / download_with_config pre-flight the gate and return FetchError::Auth with the license URL before any transfer starts. The library-level inspect functions surface the underlying HTTP 401 / 403 as FetchError::Http instead — note that the Hub serves a gated repo’s metadata publicly, so file listings succeed while content requests fail. The hf-fm CLI upgrades such inspect / diff failures into the same gated-model diagnosis the download pre-flight emits (v0.10.5).

Re-exports§

pub use config::compile_glob_patterns;
pub use config::file_matches;
pub use config::has_glob_chars;
pub use config::FetchConfig;
pub use config::FetchConfigBuilder;
pub use config::Filter;
pub use discover::DiscoveredFamily;
pub use discover::GateStatus;
pub use discover::ModelCardMetadata;
pub use discover::SearchResult;
pub use download::DownloadOutcome;
pub use error::FetchError;
pub use error::FileFailure;
pub use inspect::AdapterConfig;
pub use inspect::ModelConfig;
pub use plan::download_plan;
pub use plan::DownloadPlan;
pub use plan::FilePlan;
pub use progress::ProgressEvent;
pub use progress::ProgressReceiver;

Modules§

cache: HuggingFace cache directory resolution, model family scanning, disk usage, and integrity verification.
cache_layout: Centralized hf-hub cache path construction.
checksum: SHA256 checksum verification for downloaded files.
config: Configuration for model downloads.
discover: Model family discovery and search via the HuggingFace Hub API.
download: Download orchestration for HuggingFace model repositories.
error: Error types for hf-fetch-model.
inspect: Tensor-file header inspection (local and remote).
plan: Download plan: metadata-only analysis of what needs downloading.
progress: Progress reporting for model downloads.
repo: Repository file listing via the HuggingFace API.

Functions§

build_client: Builds a reqwest::Client with auth token, user-agent, and 30-second TCP connect timeout.
download: Downloads all files from a HuggingFace model repository.
download_blocking: Blocking version of download() for non-async callers.
download_file: Downloads a single file from a HuggingFace model repository.
download_file_blocking: Blocking version of download_file() for non-async callers.
download_files: Downloads all files from a HuggingFace model repository and returns a filename → path map.
download_files_blocking: Blocking version of download_files() for non-async callers.
download_files_with_config: Downloads files from a HuggingFace model repository using the given configuration and returns a filename → path map.
download_files_with_config_blocking: Blocking version of download_files_with_config() for non-async callers.
download_with_config: Downloads files from a HuggingFace model repository using the given configuration.
download_with_config_blocking: Blocking version of download_with_config() for non-async callers.
download_with_plan: Downloads files according to an existing DownloadPlan.
download_with_plan_blocking: Blocking version of download_with_plan() for non-async callers.