Module http_range 

HTTP Range Read + Seek substrate for remote inspection.

RangeReader adapts a byte-range transport into std::io::Read + std::io::Seek, so format parsers that consume a reader (e.g. anamnesis::inspect_npz_from_reader) can inspect a remote file’s metadata without downloading it. The transport is abstracted behind RangeFetcher; production code uses HttpRangeFetcher (HTTP Range requests against the HuggingFace CDN), and unit tests use an in-memory fetcher — the buffering, budgeting, and Seek semantics are exercised entirely offline.

Access pattern optimisations

Tuned for archive-directory reads (ZIP central directory + per-entry headers — the NPZ / PTH layout):

• Tail prefetch — the first read landing in the last TAIL_PREFETCH_BYTES of the file fetches that whole tail once and caches it. The ZIP end-of-central-directory scan and the central directory itself are then served from memory.
• Read-ahead — short sequential reads are coalesced into READAHEAD_BYTES-sized window fetches, so parsing a 30-byte local header followed by a ~120-byte NPY header costs one request, not two.
• No re-fetch — reads covered by the cached tail or the current window never re-issue a request.

Safety budgets

The reader enforces two hard caps — MAX_RANGE_REQUESTS requests and MAX_TRANSFER_BUDGET bytes fetched — so a pathological or adversarial archive layout (a seek storm, an absurdly large central directory) fails fast with a clear error instead of degenerating into a full download. RangeReader::with_limits overrides the defaults for callers with different budgets.

Error channel

Read / Seek must return std::io::Error, which flattens the crate’s typed FetchError. The reader therefore stores the most recent fetch failure; after a failed parse, RangeReader::take_last_error recovers the typed error (e.g. an HTTP 403 that the CLI upgrades into a gated-repo diagnosis).

Structs

HttpRangeFetcher

HTTP Range transport against a HuggingFace-hosted file.

RangeReader

Read + Seek over a RangeFetcher, with tail caching, read-ahead, and hard safety budgets.

RangeStats

Transfer statistics for one RangeReader lifetime.

Constants

MAX_RANGE_REQUESTS

Default cap on the number of range fetches over the reader’s lifetime.

MAX_TRANSFER_BUDGET

Default cap on total bytes fetched over the reader’s lifetime.

READAHEAD_BYTES

Minimum window size for a fetch serving a sequential read.

TAIL_PREFETCH_BYTES

Size of the cached file tail fetched by the first read near end-of-file.

Traits

RangeFetcher

A transport that serves byte ranges of one remote file.

Type Aliases

HttpRangeReader

RangeReader over HttpRangeFetcher — the production remote-inspect substrate.