# Frequently Asked Questions
<!-- Last updated: 2026-09-16, hf-fm v0.12.1 (inspect --group-by, quants --fits, --cache-headers) -->
<!--
STYLE CONVENTIONS for editing this FAQ — keep growth consistent.
1. Tone: conversational, matching the project's README voice. Address the
reader as "you". Prefer short paragraphs over bullet points.
2. Question format: "### How do I …?" or "### What is …?" as the heading.
Use natural-language questions — GitHub's anchor generator produces
usable slugs from them (e.g. "#how-do-i-install-it"). Keep them in
the contents list too.
3. Answer length: 2–4 sentences, plus at most one small code block with
a concrete command. Anything longer is a tutorial, not an FAQ entry —
link out instead.
4. Shell context: when showing env vars, give both variants side by
side — `HF_TOKEN=… hf-fm …` for bash/zsh and
`$env:HF_TOKEN="…"; hf-fm …` for PowerShell — so Windows users
are not left guessing. (See CLAUDE.md for the project's shell policy.)
5. "MSRV" should be spelled out the first time as "Minimum Rust Version
(MSRV)"; the acronym is OK on reuse.
6. Freshness marker: update the "Last updated" date and version at the
top whenever any answer text changes — not for typo fixes or new
entries that don't touch existing answers.
7. Scope: answer questions about features that actually ship today.
Do not pre-document unshipped work — those get dedicated docs when
they land.
8. Grouping: if a section grows past ~5 entries, consider splitting it.
If an entry grows past ~6 sentences, consider promoting it to
docs/tutorials/ or docs/case-studies/ and linking from here.
-->
A living list of the questions we and our early users have actually run into. If your question is not here, please open an issue on [GitHub](https://github.com/mi-for-the-rust-of-us/hf-fetch-model/issues) — we add entries as real questions arrive.
## Contents
- [About hf-fetch-model](#about-hf-fetch-model)
- [What is hf-fm? How does it differ from `huggingface-cli`?](#what-is-hf-fm-how-does-it-differ-from-huggingface-cli)
- [How does it differ from `safetensors_explorer`?](#how-does-it-differ-from-safetensors_explorer)
- [Why the two binary names, `hf-fetch-model` and `hf-fm`?](#why-the-two-binary-names-hf-fetch-model-and-hf-fm)
- [Is it stable? What does a `0.9.x` version number mean?](#is-it-stable-what-does-a-09x-version-number-mean)
- [Installation and authentication](#installation-and-authentication)
- [How do I install it? What is the Minimum Rust Version?](#how-do-i-install-it-what-is-the-minimum-rust-version)
- [How do I upgrade hf-fm? Why does `cargo install` silently keep the old version?](#how-do-i-upgrade-hf-fm-why-does-cargo-install-silently-keep-the-old-version)
- [Is the `hf-fm` binary code-signed? Why did my antivirus flag it?](#is-the-hf-fm-binary-code-signed-why-did-my-antivirus-flag-it)
- [What is the `cli` feature and do I need it?](#what-is-the-cli-feature-and-do-i-need-it)
- [How do I pass a HuggingFace token? Why does a gated model fail?](#how-do-i-pass-a-huggingface-token-why-does-a-gated-model-fail)
- [Discovery — finding what to inspect or download](#discovery--finding-what-to-inspect-or-download)
- [A repo has many `.safetensors` files — how do I pick one to inspect?](#a-repo-has-many-safetensors-files--how-do-i-pick-one-to-inspect)
- [A repo has many `.safetensors` files — can I pick one interactively?](#a-repo-has-many-safetensors-files--can-i-pick-one-interactively)
- [How do I read a small config or README file from a repo without downloading it?](#how-do-i-read-a-small-config-or-readme-file-from-a-repo-without-downloading-it)
- [How do I see a model's tensor names without downloading it?](#how-do-i-see-a-models-tensor-names-without-downloading-it)
- [How do I compare two HuggingFace models structurally?](#how-do-i-compare-two-huggingface-models-structurally)
- [How do I compare two models' architecture, not just their tensors?](#how-do-i-compare-two-models-architecture-not-just-their-tensors)
- [How do I know if a model fits on my GPU?](#how-do-i-know-if-a-model-fits-on-my-gpu)
- [What fraction of a GGUF file is the MoE expert weights?](#what-fraction-of-a-gguf-file-is-the-moe-expert-weights)
- [Which quant of a model fits my GPU?](#which-quant-of-a-model-fits-my-gpu)
- [How reliable is `quants`'s sibling-repo discovery?](#how-reliable-is-quantss-sibling-repo-discovery)
- [How do I list only the weight files in a repo, not the tokenizer and README?](#how-do-i-list-only-the-weight-files-in-a-repo-not-the-tokenizer-and-readme)
- [Why did `list-files` say "509 GiB total" for a repo I'd never download in full?](#why-did-list-files-say-509-gib-total-for-a-repo-id-never-download-in-full)
- [How do I see what is already cached locally?](#how-do-i-see-what-is-already-cached-locally)
- [Cache location and management](#cache-location-and-management)
- [Where does hf-fm store downloaded files? Is the layout compatible with Python `huggingface_hub`?](#where-does-hf-fm-store-downloaded-files-is-the-layout-compatible-with-python-huggingface_hub)
- [My disk is getting full — which models are taking the most space?](#my-disk-is-getting-full--which-models-are-taking-the-most-space)
- [What is a `.chunked.part` file, and is it safe to delete?](#what-is-a-chunkedpart-file-and-is-it-safe-to-delete)
- [Downloading large files on slow connections](#downloading-large-files-on-slow-connections)
- [Why does my download keep timing out, and how do I extend the budget?](#why-does-my-download-keep-timing-out-and-how-do-i-extend-the-budget)
- [My download was interrupted — do I have to start over?](#my-download-was-interrupted--do-i-have-to-start-over)
- [Errors and unexpected output](#errors-and-unexpected-output)
- [Which file formats can `inspect` read?](#which-file-formats-can-inspect-read)
- [Is it safe to `inspect` a `.pth` file from a repo I don't trust?](#is-it-safe-to-inspect-a-pth-file-from-a-repo-i-dont-trust)
- [I got a `checksum mismatch` error — what do I do?](#i-got-a-checksum-mismatch-error--what-do-i-do)
- [Why does `inspect` say `Source: remote (N range requests, X fetched)`?](#why-does-inspect-say-source-remote-n-range-requests-x-fetched)
- [How do I avoid re-fetching the same remote header on every inspect call?](#how-do-i-avoid-re-fetching-the-same-remote-header-on-every-inspect-call)
- [Why didn't my pipeline catch a download failure?](#why-didnt-my-pipeline-catch-a-download-failure)
---
## About hf-fetch-model
### What is hf-fm? How does it differ from `huggingface-cli`?
`hf-fetch-model` is a Rust CLI and library for downloading, inspecting, and comparing HuggingFace models. It overlaps with Python's `huggingface-cli` on the basics (fetch a repo, list files, manage the cache) but diverges in three places:
1. it downloads large files with many parallel HTTP connections,
2. it can read a model's tensor-name/dtype/shape metadata **without downloading the weights**, via narrow HTTP Range requests against the file header — `.safetensors`, `.npz`, `.gguf`, and `.pth`, all remotely,
3. it ships as a standalone binary with no Python dependency. It writes to the same cache directory as Python `huggingface_hub`, so the two tools coexist happily.
### How does it differ from `safetensors_explorer`?
[`safetensors_explorer`](https://github.com/EricLBuehler/safetensors_explorer) by Eric Buehler is a Rust TUI for exploring safetensors and GGUF files — full-screen interactive browsing, fuzzy search over tensor names, directory-wide merging. It and `hf-fm inspect` are complementary rather than competing:
1. `safetensors_explorer` is an interactive **TUI** that shines at exploring a model locally with the keyboard; `hf-fm inspect` is a **CLI** that produces printable output (pipeable into other tools, pasteable into bug reports),
2. `safetensors_explorer` reads **local files only**; `hf-fm inspect` additionally reads the tensor metadata of a **remote** model via HTTP Range, before anything is downloaded,
3. `safetensors_explorer` currently covers **safetensors and GGUF**; hf-fm covers **safetensors, NumPy `.npz`, GGUF, and PyTorch `.pth`** — all four remote or cached (remote NPZ since v0.11.0, remote safetensors since v0.11.1, remote GGUF since v0.11.2, remote `.pth` since v0.11.4).
Reach for `safetensors_explorer` when you want to sit at a TUI and explore a model locally; reach for `hf-fm inspect` when you want to preview a remote model before downloading, or when you want text output you can pipe and paste.
### Why the two binary names, `hf-fetch-model` and `hf-fm`?
They are the same program. `hf-fetch-model` is the long form that appears on crates.io and shows up in `cargo install` output; `hf-fm` is the short alias you actually type. Both binaries are installed by the same `cargo install` command — pick whichever reads better in your shell history.
### Is it stable? What does a `0.9.x` version number mean?
hf-fm is production-grade on the download and inspect paths we use daily, but the `0.x` version number means we follow Cargo's SemVer interpretation: any minor bump (`0.9 → 0.10`) is allowed to break public API. Patch bumps (`0.9.6 → 0.9.7`) do not break the API — so if you are pinning a dependency, a `~0.9` or `=0.9.7` constraint is safe. The `CHANGELOG.md` always lists breaking changes under each release.
---
## Installation and authentication
### How do I install it? What is the Minimum Rust Version?
From crates.io, in one command:
```
cargo install hf-fetch-model --features cli
```
That installs both `hf-fetch-model` and `hf-fm` into `~/.cargo/bin` (or `%USERPROFILE%\.cargo\bin\` on Windows). The Minimum Rust Version (MSRV) is **1.88**, declared via the `rust-version` field in `Cargo.toml` — Cargo warns if your active toolchain is older. If `cargo install` fails complaining about the Rust version, run `rustup update stable`.
### How do I upgrade hf-fm? Why does `cargo install` silently keep the old version?
Pass `--force` to `cargo install` and then re-check the version that actually landed:
```
cargo install hf-fetch-model --features cli --force
hf-fm --version
```
Without `--force`, `cargo install` short-circuits whenever **any** version of the binary is already in `~/.cargo/bin/` — even when crates.io has a newer release. The "already installed" notice is logged to stderr at low priority and is easy to miss in a busy terminal: the install command exits `0`, looks like it succeeded, but the binary on `PATH` is unchanged. `--force` bypasses the short-circuit and always builds the latest. The companion note at [`docs/dogfooding-feedbacks/cargo-install-silent-skip.md`](dogfooding-feedbacks/cargo-install-silent-skip.md) captures the failure mode in full — including a reproduction recipe and the proposed `hf-fm --check-update` flag tracked as a future patch-release candidate.
### Is the `hf-fm` binary code-signed? Why did my antivirus flag it?
hf-fm is distributed as **source on crates.io**, not as a pre-built executable. When you run `cargo install hf-fetch-model --features cli`, Cargo compiles the binary **on your own machine** from source you can read. There is no publisher-signed `.exe` to verify because there is no download of a finished binary — the trust model is "build it yourself from open source," which is stronger than trusting a signature on someone else's build. So today the honest answer to "is it Authenticode/notarization signed?" is: it doesn't need to be, because nothing pre-compiled is shipped.
If a heuristic antivirus (Windows Defender SmartScreen, etc.) flags the freshly-built `hf-fm.exe` or a script that launches it, that is almost always a **behavioral false positive** — multi-connection downloaders and process-launching scripts pattern-match to loader malware. It is not specific to hf-fm. Two clean responses: build in a directory your AV is told to trust, or — if you want Windows to trust the binary itself — code-sign it. For purely local use, a self-signed certificate you import into your own Trusted Publishers store is enough:
```powershell
$cert = New-SelfSignedCertificate -Type CodeSigningCert -Subject "CN=hf-fm (local)" -CertStoreLocation Cert:\CurrentUser\My
Set-AuthenticodeSignature -FilePath "$env:USERPROFILE\.cargo\bin\hf-fm.exe" -Certificate $cert -TimestampServer http://timestamp.digicert.com
```
A self-signed certificate is trusted only on machines where you install it; it does nothing for other users. **Publicly-trusted** signing (the kind that satisfies SmartScreen for everyone) requires a CA-issued certificate whose key lives on FIPS-140 hardware — only worth pursuing if/when hf-fm starts shipping pre-built GitHub Release binaries. The likely path then is the [SignPath Foundation](https://signpath.org/), which provides free CI-integrated Authenticode signing to qualifying open-source projects (OSI license, actively maintained, repo-owned builds — hf-fm qualifies on all stated criteria). Until pre-built binaries ship, building from source remains the recommended and verifiable install.
### What is the `cli` feature and do I need it?
hf-fm ships as a **library crate by default** — no command-line binaries unless you ask for them. The `cli` feature pulls in `clap`, `tracing-subscriber`, and the progress bar dependencies needed to build the `hf-fetch-model` and `hf-fm` executables. You need it when running `cargo install`; you do **not** need it when adding hf-fetch-model as a dependency to your own Rust project — in that case, use the default library-only build and call `FetchConfig::builder()` directly.
### How do I pass a HuggingFace token? Why does a gated model fail?
Either pass `--token <value>` on the command line, or set the `HF_TOKEN` environment variable once and forget about it — every subcommand reads it automatically. Gated models (like Meta's Llama releases or Google's Gemma family) require both a valid token and that you have accepted the licence on the model's HuggingFace page. When either is missing, `download`, `inspect`, and `diff` report an `authentication failed: <repo> is a gated model …` error carrying the license URL and token guidance (`download` checks the gate up front, since v0.9.3; `inspect` and `diff` diagnose the underlying 401/403 from their Range requests, since v0.10.5 — `diff` names whichever side is gated).
Three traps worth knowing:
1. **A gated repo's metadata is public.** `list-files` and `inspect --list` fill in sizes and SHA256 hashes with no token at all — only *content* requests (downloads, header Range requests) hit the gate. A successful listing is not proof of access.
2. **Each gated family is licensed separately.** Accepting the Llama 3.2 license grants nothing for Llama 3.1 — request access on the exact repo you need.
3. **Fine-grained tokens need the gated-repo permission.** A fine-grained token without *"Read access to contents of all public gated repos you can access"* gets 403 even after the license is accepted; classic "read" tokens don't have this failure mode.
One useful consequence of trap 1: you can answer *"does this gated model even fit my GPU?"* **before** accepting any license. The public listing already shows the `.safetensors` sizes (their sum ≈ the weight bytes), and popular gated models usually have byte-identical ungated mirrors — verify with `list-files` on both and compare the SHA256 columns; if every LFS hash matches, the mirror *is* the model. Then `hf-fm inspect <mirror> --check-gpu --context N` gives the full header-precise verdict, license-free. Example: `NousResearch/Meta-Llama-3.1-8B` mirrors `meta-llama/Llama-3.1-8B` hash-for-hash.
```
# bash / zsh
HF_TOKEN=hf_xxx hf-fm meta-llama/Llama-3.2-1B
# PowerShell
$env:HF_TOKEN="hf_xxx"; hf-fm meta-llama/Llama-3.2-1B
```
---
## Discovery — finding what to inspect or download
### A repo has many `.safetensors` files — how do I pick one to inspect?
Use `inspect --list` to see them all with sizes, then call `inspect <repo> <n>` with the index you want:
```
hf-fm inspect Qwen/Qwen2.5-Coder-7B-Instruct --list
# → 1 model-00001-of-00004.safetensors 4.54 GiB
# 2 model-00002-of-00004.safetensors 4.59 GiB
# …
hf-fm inspect Qwen/Qwen2.5-Coder-7B-Instruct 2 --tree
```
Indices are stable for as long as the repository does not change remotely — for scripted reproducibility, pass the `--revision <sha>` that `--list` prints in its header to both commands. Since v0.10.5, the listing covers every tensor format `inspect` can read (`.safetensors` / `.gguf` / `.npz` / `.pth`), so a GGUF-only repo lists and index-resolves the same way. Prefer one command over two? The next question covers the interactive `--pick` flag, which shares the same numbered universe.
### A repo has many `.safetensors` files — can I pick one interactively?
Yes — `--pick` (since v0.10.5) collapses the two-step `--list` → `inspect <repo> <n>` workflow into one command:
```
hf-fm inspect little-lake-studios/demoncore-flux demonCORE --pick --dtypes
# Multiple tensor files match "demonCORE" in little-lake-studios/demoncore-flux:
# 1 transformer/demonCORENSFW_fluxV11.safetensors 15.40 GiB
# 2 transformer/demonCORESFWNSFW_fluxV12.safetensors 15.40 GiB
# 3 transformer/demonCORESFWNSFW_fluxV13.safetensors 15.40 GiB
# Pick [1..3]: 3
# Resolving to transformer/demonCORESFWNSFW_fluxV13.safetensors
```
Under `--pick`, the positional argument is a **case-insensitive substring** filter, never a numeric index: when exactly one file matches, hf-fm skips the prompt and resolves directly (printing `Resolving to <name>` on stderr); with no positional at all, the picker offers every supported tensor file. It composes with every rendering flag (`--tree`, `--dtypes`, `--filter`, `--limit`, `--check-gpu`, `--json`) — the table and prompt go to **stderr**, so `--pick --json > out.json` still writes clean JSON to the file. Pressing Enter on an empty line (or Ctrl-D / Ctrl-Z) cancels with a non-zero exit code.
`--pick` requires an interactive terminal (stdin and stderr attached). In scripts and CI, use `--list` + the numeric index instead — the two workflows coexist and share the same alphabetically-sorted file universe, so `#3` means the same file in both.
### How do I read a small config or README file from a repo without downloading it?
Use `peek` (v0.11.5) — it reuses `inspect`'s HTTP Range substrate, but for everything `inspect` doesn't cover: `config.yaml`, `README.md`, license texts, `.gz`-compressed sidecars. It never parses tensor headers (`inspect` rejects `.safetensors`/`.gguf`/`.npz`/`.pth` and points you back here for anything else):
```
hf-fm peek julien-c/dummy-unknown README.md
hf-fm peek julien-c/dummy-unknown config.json --head 5
```
`--head N` / `--tail N` bound the read (lines by default, `--bytes` for raw byte counts); `--tail --bytes` is one cheap Range-from-end request, `--tail` lines does a bounded backward scan. A `.gz`-suffixed file is transparently decoded (composes with `--head`, not with `--tail` — gzip is sequential, so decompress with `--head` and pipe through `tail` instead):
```
hf-fm peek bluelightai/clt-qwen3-1.7b-base-20k features/index.json.gz --head 5 --bytes
```
`--max <SIZE>` (default `10 MiB`) is the safety cap: an unbounded `peek` of a file over the cap is **rejected before any content byte is fetched**, not truncated to raw output — accidentally peeking a multi-gigabyte weight file gets a clear error pointing at `inspect` instead of binary garbage on your terminal. There's no `--cached` flag; the cached equivalent is `cat $(hf-fm cache path <repo>)/<file>` (`Get-Content` on PowerShell).
### How do I see a model's tensor names without downloading it?
Run `hf-fm inspect <repo>` with no filename for a per-file (or, on sharded repos, per-shard) rollup — tensor *counts* and parameters. To see the tensor *names*, either name a specific file (or an index from `--list`), or add `--filter "<substring>"`, which lists the matching names nested under each shard/file (e.g. `--filter "layers.0."` for everything in block 0; the match is case-insensitive). Internally, hf-fm fetches only the JSON header via an HTTP Range request — for a typical 2 GiB safetensors file, you transfer maybe 70 KiB of metadata. Add `--tree` for the hierarchical view that groups numeric layers (`layers.[0..27] (×28)`), or `--dtypes` for a dtype-and-parameter summary. For a complete walkthrough on a real 4-shard model, see [Inspect before you download](tutorials/inspect-before-downloading.md).
### How do I compare two HuggingFace models structurally?
`hf-fm diff <REPO_A> <REPO_B>` classifies every tensor across both repos into four buckets — *only-in-A*, *only-in-B*, *dtype/shape differences*, and *matching* — by reading each side's safetensors headers via HTTP Range (no weight data downloaded). For scaled-sibling pairs (one model is a bigger sibling of the other in the same family) the per-tensor output is dominated by the extra-layer wall, so reach for `--dtypes` instead:
```
$ hf-fm diff openai/gpt-oss-20b openai/gpt-oss-120b --dtypes
A: openai/gpt-oss-20b
B: openai/gpt-oss-120b
Dtype A Tensors A Size B Tensors B Size Δ Size
U8 192 18.91 GiB 288 113.46 GiB +94.55 GiB
BF16 630 6.72 GiB 942 8.07 GiB +1.35 GiB
──────────────────────────────────────────────────────────────
A: 822 tensors, 25.63 GiB | B: 1230 tensors, 121.54 GiB | Δ: +408 tensors, +95.90 GiB
```
This is a side-by-side per-dtype histogram with a signed Δ Size column. Same dtype mix on both sides plus proportional scaling = scaled siblings (same architecture, different size). A dtype present in only one side, or a wildly disproportionate Δ Size ratio across dtypes, would point at an architectural variant rather than a clean scale-up. Composes with `--filter` (histograms aggregate over filtered tensors only) and `--json`. The complementary text mode `diff` (without `--dtypes`) remains useful for short tensor-level inspections — see `hf-fm diff --help`.
For deeper structural analysis on the per-tensor list (when `--dtypes` says "same dtype mix but I want to see *which* tensors differ"), the JSON output now ships a `byte_count` field on every entry. Pipe it through `jq` to collapse `only_a` / `only_b` entries by name pattern — useful when one side has 200 extra layers and you want the *kinds* of new tensors, not the raw 200-row list:
```bash
hf-fm diff org/model-A org/model-B --json \
| jq -r '
.only_b
| group_by(.name | gsub("[0-9]+"; "{N}"))
| map({
pattern: (.[0].name | gsub("[0-9]+"; "{N}")),
tensors: length,
bytes: (map(.b.byte_count) | add),
})
| sort_by(-.bytes)
| .[] | "\(.pattern) \(.tensors) \(.bytes) bytes"
'
```
For a scaled-sibling pair this collapses `model.layers.0.self_attn.q_proj.weight`, `model.layers.1.self_attn.q_proj.weight`, … into a single `model.layers.{N}.self_attn.q_proj.weight` line with a count and a summed-byte total. The same recipe with `.only_a` swapped in does the symmetric job.
That numeric-segment heuristic is now also a built-in flag: `--collapse` groups only-A / only-B / dtype-shape-differences into `Pattern / Tensors / Bytes` tables directly, no `jq` needed — reach for it first. The `jq` recipe above is still worth keeping around for cases `--collapse` doesn't cover: it only groups by digit-run substitution, independently per section (no cross-referencing between only-A and only-B, no expert-routing-aware grouping), so a different heuristic on your own pair is still a `jq` filter away.
### How do I compare two models' architecture, not just their tensors?
`hf-fm diff` and `--dtypes`/`--collapse` above answer *what changed in the tensors*; `hf-fm diff-config <REPO_A> <REPO_B>` answers *why* — a field-by-field diff of `config.json`'s architecture fields (layer count, hidden size, GQA/sliding-window/hybrid-layout fields, and more):
```
$ hf-fm diff-config openai/gpt-oss-20b openai/gpt-oss-120b
A: openai/gpt-oss-20b
B: openai/gpt-oss-120b
Field A B
num_hidden_layers 24 36
layer_types sliding_attention, full_attention, sliding_attention, full_a… sliding_attention, full_attention, sliding_attention, full_a…
2 of 23 fields differ
```
Reach for this when `diff --dtypes` shows a size jump and you want the architectural cause without eyeballing two raw `config.json` files side by side. Add `--all` to see matching fields too (default is differences-only); `--json` always carries every field with a `differs` flag, regardless of `--all`.
### How do I know if a model fits on my GPU?
Pass `--check-gpu` for a weights-vs-VRAM verdict, and add `--context N` for the full picture including the KV cache:
```
hf-fm inspect meta-llama/Llama-3.2-3B --cached --check-gpu --context 32768
```
`--check-gpu` alone reads the device's total / free / used VRAM via [`hypomnesis`](https://crates.io/crates/hypomnesis) (NVML on Linux/Windows, DXGI on Windows; `nvidia-smi` fallback), sums the model's weight bytes across every shard, and prints a one-line `✓ X.YZ GiB headroom` / `✗ short by X.YZ GiB` verdict. Default device is `0`; pass `--check-gpu 1` on a multi-GPU box. Works on the cached and network paths identically. On a system with no NVIDIA GPU the verdict reports `unavailable — <reason>` and the command still exits 0 — `--check-gpu` is informational, never a gate.
**`--context N` (v0.10.4) makes it real.** Weights are the easy part; on a consumer card the KV cache is what decides whether a long context fits. `--context N` reads the model's `config.json` and computes the KV bytes at sequence length `N`, then measures the fit against `weights + KV`:
```
Model weights: 5.98 GiB (BF16, 3.21B params)
KV cache @ ctx=32768: 3.50 GiB (BF16)
Total: 9.48 GiB (weights + KV)
Fit: ✓ 4.20 GiB headroom (weights + KV; runtime extra)
Spilling: not sampled (platform supports detection)
```
The estimate is **parameter-driven, not a per-model lookup table** — it applies the universal formula `2 × layers × kv_heads × head_dim × N × dtype_bytes` to the model's actual architecture integers, so a model hf-fm has never seen computes correctly. It is architecture-aware where the simple formula breaks:
- **GQA** (Llama-3, Mistral): uses `num_key_value_heads`, not the query-head count (often a 4×+ difference).
- **Sliding window** (Mistral, Phi): KV caps at the window. Gemma-2 / Gemma-3 mix local and global layers, which are *blended* (global layers at full context + local layers at the window).
- **Hybrid Mamba/attention** (Granite-4, Nemotron-H, Bamba, Qwen3-Next): KV applies only to the few attention layers, and a separate `Recurrent state` line reports the fixed Mamba2 state (which does **not** grow with context). This is why a hybrid fits far more context than a same-size transformer.
- **MLA** (DeepSeek): the naive formula overestimates ~10×, so it is **skipped** with a note rather than printing a wrong number.
`--context` requires `--check-gpu`, and composes with `--json` (the `gpu_check` object gains a `kv_cache` sub-object and `model.total_bytes`).
**Known limitations** — all flagged in the output, none silent:
- **MLA is skipped, not estimated.** DeepSeek-V2/V3 print `KV cache: skipped (MLA / latent attention — naive estimate unreliable)` and fall back to the weights-only verdict.
- **Mixed sliding-window is approximate** (within a few percent): the Gemma blend models the *count* of local vs global layers, not their exact positions in the stack.
- **KV dtype is assumed equal to the activation dtype.** The KV element size comes from the config's `torch_dtype` (bf16 / fp16 = 2 bytes), independent of weight quantization. If you run an FP8 / Q4 KV cache to fit more context, the real figure is smaller — so treat the reported number as a safe upper bound.
- **Non-Mamba2 recurrent state is excluded.** For Qwen3-Next (Gated DeltaNet) and Jamba (Mamba1) the *attention* KV is correct, but the recurrent state is labeled `excluded (small, constant)` rather than computed — it is tens of MiB and constant in context, so it never flips a consumer-GPU verdict.
### What fraction of a GGUF file is the MoE expert weights?
Pass `--group-by` with a glob matching the expert tensor names:
```
hf-fm inspect poolside/Laguna-XS-2.1-GGUF Q4_K_M.gguf --group-by 'blk.*.ffn_*_exps.weight'
```
This buckets every tensor into MATCHED / OTHER by name and prints byte totals, percentages, and — when the matched names carry a single, unambiguous numeric layer index — a `per-MoE-layer expert cost` line. That per-layer figure is exactly what CPU-expert-offload planning (`llama.cpp`'s `--n-cpu-moe`) needs: a large MoE checkpoint that looks too big for your VRAM at first glance can still fit once you know only a fraction of it is expert weight that can live in system RAM. This surfaced from a real [dogfooding session](dogfooding-feedbacks/hf-fm-dogfooding-vram-fit-laguna-session.md) sizing quant candidates by hand with `awk`.
### Which quant of a model fits my GPU?
Pass `--fits` to `quants`:
```
hf-fm quants poolside/Laguna-XS-2.1 --fits 16GiB --reserve 2.5GiB
```
This aggregates the base model's quant sibling repos into one table, sorted by size, and adds a `RESIDENT`/`PLAN` column pair. Candidates that already fit under the budget render `full GPU` with no network cost; only over-budget `.gguf` files are inspected to compute a `--n-cpu-moe N` offload plan, so a checkpoint that looks too big at first glance can still surface as viable once its `MoE` expert tensors are known to be offloadable. `--reserve` carves out headroom (KV cache, runtime) from the budget before the comparison. See the [tutorial](tutorials/pick-a-quant-that-fits.md) for the full walkthrough.
### How reliable is `quants`'s sibling-repo discovery?
There is no HuggingFace Hub endpoint for "find the quant siblings of this repo", so `quants` combines two imperfect signals: a naming match (any repo whose ID contains the base model's short name) builds the candidate pool, and — for `.gguf` candidates — the file's own metadata backlink (`general.source.url` / `general.base_model.*.repo_url`) confirms it when present and checkable. A candidate is only ever *excluded* when a backlink explicitly names a different repo; no backlink, or a failed backlink check (network error, a gated repo), still lists the candidate, just without the `verified` mark — a transient failure should never hide a real candidate. In practice this means naming matches occasionally include unrelated repos that happen to share the base model's name (a full-precision mirror, an unrelated fine-tune) — the stderr summary line's verified count and each row's provenance are the signal to sanity-check before trusting a result.
### How do I list only the weight files in a repo, not the tokenizer and README?
Use `list-files` with a `--preset` matching the weight format:
```
hf-fm list-files google/gemma-2-2b-it --preset safetensors
hf-fm list-files google/gemma-scope-2b-pt-transcoders --preset npz
```
The presets bundle the weight extension plus the common config files (`*.json`, `*.txt`, and for `npz` the `config.yaml` GemmaScope uses). Available presets are `safetensors`, `gguf`, `npz`, `pth`, and `config-only` — `hf-fm list-files --help` shows the full list.
### Why did `list-files` say "509 GiB total" for a repo I'd never download in full?
Some `.gguf` repos hold N *mutually-exclusive* quantizations of the same model (`Q4_K_M.gguf`, `Q5_K_M.gguf`, `Q8_0.gguf`, ...) rather than N shards of one file — summing their sizes implied a single download nobody would make. `list-files`, `du <repo>`, and `search --show size` now detect this shape (versus a genuinely sharded file, where the total is correct because every shard is needed) and show a `min to max (mutually exclusive quants)` range instead:
```
hf-fm list-files bartowski/gemma-2-2b-it-GGUF --preset gguf
# 11 files, 1.30 GiB to 9.74 GiB (mutually exclusive quants)
```
`list-files --json` and `du <repo> --json` keep `total_bytes` as the plain sum (still well-defined, just not the useful number here) and add `quant_alternatives`/`size_min`/`size_max` alongside it — an additive schema change, so existing consumers reading only `total_bytes` are unaffected.
### How do I see what is already cached locally?
Four views, in order of detail:
- `hf-fm list-families` — grouped by model architecture (`gemma2`, `llama`, `qwen2`, …), one repo per line under each family. Shows which cache directory you are looking at on the first line.
- `hf-fm du` — one row per repo with disk usage and a partial-download marker, sorted largest first. Type `hf-fm du <N>` to drill into the Nth repo and see its files.
- `hf-fm du --tree` — hierarchical view of every cached repo and its files in a single box-drawing tree, with sizes right-aligned across all rows. Composes with `--age` to add a last-modified column on each repo branch.
- `hf-fm status <repo>` — per-file completeness status for one repo.
---
## Cache location and management
### Where does hf-fm store downloaded files? Is the layout compatible with Python `huggingface_hub`?
Yes, fully compatible. hf-fm writes to `~/.cache/huggingface/hub/` by default (or `$HF_HOME/hub/` when the environment variable is set), using the same `models--<org>--<name>/` directory layout, the same `blobs/` and `snapshots/` subfolders, and the same `refs/` ref files that Python's `huggingface_hub` produces. You can download a model with Python and inspect it with hf-fm, or vice versa, without either tool noticing the other. On Windows the path is `%USERPROFILE%\.cache\huggingface\hub\`.
### My disk is getting full — which models are taking the most space?
Run `hf-fm du` for a size-sorted summary. Each row has a `#` index you can pass back to the command to see that repo's files one level deeper (`hf-fm du 3`), and `du --age` adds a "last modified" column that is handy for spotting models you downloaded once months ago and never touched again. When you have identified what to remove, use `hf-fm cache delete <repo-id>` (repo ID or the numeric index from `du`); it prompts for confirmation unless you pass `--yes`. For bulk reclaiming — age-based or size-budget eviction with a `--dry-run` preview — the [Clean up before your disk fills](tutorials/clean-up-before-your-disk-fills.md) tutorial walks the whole `du` → `status` → `cache gc` workflow on a real 592 GiB cache.
### What is a `.chunked.part` file, and is it safe to delete?
`.chunked.part` files are temporary staging files for multi-connection downloads — hf-fm writes downloaded bytes there before renaming them into the final blob. From v0.9.8 onwards they also persist across interruptions so the next invocation can resume from them, paired with a small `.chunked.part.state` JSON sidecar that tracks per-chunk progress. They are safe to delete when you have abandoned a download for good — run `hf-fm cache clean-partial` for a prompted cleanup (add `--dry-run` to preview, `--yes` to skip confirmation); the sweep removes the partial and its sidecar together. A repo with a stale partial also shows up as a `●` marker in `hf-fm du`.
---
## Downloading large files on slow connections
### Why does my download keep timing out, and how do I extend the budget?
By default hf-fm gives each file a 300-second budget — fine for typical multi-GiB safetensors at typical home-broadband speeds, but it can run out before a 10 GiB file finishes if your effective throughput is below ~35 MiB/s. Pass `--timeout-per-file-secs <N>` to extend it; `1800` (30 minutes) is a sensible value for files in the 5–15 GiB range on slower links. The companion `--timeout-total-secs` flag is an overall wall-clock budget for the whole invocation — it bounds a multi-file batch *and* a single large file (including `download-file`), elapsing whichever comes first against the per-file limit. (Before v0.10.5 the total budget was only checked between files, so it silently failed to interrupt a single dominant download; it is now a hard cap on in-flight transfers too.)
```
hf-fm google/gemma-4-E2B-it --preset safetensors --timeout-per-file-secs 1800
```
### My download was interrupted — do I have to start over?
No. As of v0.9.8, hf-fm preserves the partial `.chunked.part` file plus a small `.chunked.part.state` sidecar that records per-chunk progress; the next time you run the same command, each parallel chunk picks up from where it stopped. This works across timeouts, Ctrl-C, and crashes — as long as the file on the remote did not change (the etag is verified before resuming). If the etag does not match, hf-fm starts fresh and tells you so.
---
## Errors and unexpected output
### Which file formats can `inspect` read?
Four tensor formats, all remote via HTTP Range or cached: `.safetensors` (remote since v0.11.1), NumPy `.npz` (remote since v0.11.0), `.gguf` (remote since v0.11.2), and PyTorch `.pth` (remote since v0.11.4 — only the `data.pkl` pickle stream inside the `ZIP` archive is fetched, never the tensor-data files). One error points at the edge of that support: an unsupported extension (`.bin`, etc.) gives `hf-fm inspect supports .safetensors, .gguf, .npz, or .pth (got .bin for …)`. On a repo you have not fetched yet, `hf-fm list-files <repo>` shows what is available first. For everything else in a repo — `config.yaml`, `README.md`, `.gz` sidecars — use `peek` (v0.11.5) instead; see [the Discovery entry above](#how-do-i-read-a-small-config-or-readme-file-from-a-repo-without-downloading-it).
### Is it safe to `inspect` a `.pth` file from a repo I don't trust?
Yes — hf-fm never executes arbitrary code from a `.pth` file, cached or remote. PyTorch's `.pth` format is a ZIP archive containing a pickle stream (`data.pkl`); Python's own `pickle.load()` can execute arbitrary code embedded in that stream, which is the well-known reason `torch.load(weights_only=False)` is considered unsafe on untrusted files. hf-fm never touches Python's pickle loader — the `anamnesis` parser crate implements its own minimal pickle interpreter (~36 opcodes) with an explicit `GLOBAL` allowlist: only PyTorch tensor-reconstruction calls (`torch._utils._rebuild_tensor_v2` and friends) and `collections.OrderedDict` are permitted, and any other reference is rejected before it can do anything. The parser is additionally hardened against unguarded-allocation denial of service — a crafted file that declares an astronomical pickle size or tensor count fails fast with a clear error instead of driving a multi-GiB allocation. `inspect` reads metadata only — no tensor data is ever materialised, cached or remote — so the blast radius of a malicious `.pth` is "the parse returns an error," not "arbitrary code runs."
### I got a `checksum mismatch` error — what do I do?
A `checksum mismatch` means the file's computed SHA256 does not match the hash HuggingFace's API reported. Most of the time this is caused by a truncated download — delete the local file (or run `hf-fm cache delete <repo>` for the whole repo) and retry; the multi-connection download path will re-fetch it cleanly. If the mismatch repeats on a fresh download, that is genuinely unusual — open an issue on [GitHub](https://github.com/mi-for-the-rust-of-us/hf-fetch-model/issues) with the repo ID, filename, and the error message.
### Why does `inspect` say `Source: remote (N range requests, X fetched)`?
Remote `.safetensors`, `.npz`, and `.gguf` inspect (v0.11.1, v0.11.0, and v0.11.2 respectively) all ride the same `HttpRangeReader` substrate, and the line reports the *measured* cost of that run rather than a fixed count — the on-screen proof that `inspect` read metadata, not weights. The reader enforces hard budgets (256 requests / 32 MiB per inspect), so even a hostile or corrupted file cannot silently turn an inspect into a full download. When the file is already in your local cache, the line reads `Source: cached` instead and there are no HTTP requests at all — distinct from `Source: cached header (age: ...)`, which `--cache-headers` (below) produces for a *remote* file whose already-parsed header was reused.
For `.safetensors`, the header is a little-endian `u64` length prefix followed by the `JSON` header itself, both at the very start of the file — the reader's 4 KiB read-ahead window usually covers both in a single fetch, with a second fetch only when the header is larger than that window (common on many-tensor models). Every remote path on this substrate also bakes in a fixed one-time access probe (2 requests), so a small shard reads e.g. `Source: remote (4 range requests, 8.0 KiB fetched)` (live-measured against `hf-internal-testing/tiny-random-gpt2`). For `.npz`, the requests fetch the `ZIP` central directory and the per-array `NPY` headers — e.g. `Source: remote (6 range requests, 136.0 KiB fetched)` against a 72 MiB GemmaScope archive. For `.gguf`, the requests cover the front-loaded metadata KV table and tensor-info table (both live before the tensor-data section, so a single linear scan never touches weight bytes) — e.g. `Source: remote (30 range requests, 1.75 MiB fetched)` against an 84 MiB quantized `bartowski/SmolLM2-135M-Instruct-GGUF` shard. No format on this substrate ever downloads tensor data.
### How do I avoid re-fetching the same remote header on every inspect call?
Pass `--cache-headers`:
```
hf-fm inspect bartowski/gemma-2-2b-it-GGUF gemma-2-2b-it-Q4_K_M.gguf --cache-headers
```
The first call parses the header normally and saves it to a `.hf-fm-header-cache/` sidecar next to the repo's usual cache directory, keyed on `(repo, revision, filename, etag)`. A second call against the same file reports `Source: cached header (age: 2m)` and skips the range requests entirely — a changed etag (the upstream file was updated) is a cache miss, not a stale hit. Off by default: a plain `inspect` never touches local disk without this flag, even against a repo you have never downloaded. Useful for the iterative-narrowing pattern `hf-fm quants` encourages — checking the same handful of candidates more than once while deciding.
### Why didn't my pipeline catch a download failure?
If you wrap a hf-fm command in a shell pipe like `hf-fm download-file ... 2>&1 | tail -20`, the pipeline's exit code is the **last** command's, not hf-fm's. So a hf-fm timeout or network error can be hidden behind a successful `tail`, leaving you thinking the download worked when it didn't. This is true of every CLI tool, not just hf-fm — but it bites here because long downloads invite the impulse to wrap them in `| tail` to keep the terminal tidy. hf-fm does print `error: …` lines to stderr on failure, but if you fold stderr into stdout via `2>&1` and then truncate, the failure signal lives in the tail of the output rather than the exit code.
To check hf-fm's real exit code through a pipe:
```
# bash / zsh — PIPESTATUS holds each pipeline stage's exit code
hf-fm download-file ... 2>&1 | tail -20
echo "hf-fm exit: ${PIPESTATUS[0]}"
# PowerShell — capture and inspect $LASTEXITCODE after the producer
hf-fm download-file ... 2>&1 | Select-Object -Last 20
echo "hf-fm exit: $LASTEXITCODE"
```
Or skip the pipe and check `$?` (bash/zsh) / `$LASTEXITCODE` (PowerShell) directly after the bare command — simplest when you do not actually need to truncate the output.