Skip to main content

Module format

Module format 

Source
Expand description

File-format classification from magic bytes, extension, and media type.

§Why a retriever needs this

A download manager sorts what it fetches, and the category decides real behaviour: which directory the file lands in, and whether the extension can be trusted. But it decides those things only if the classification is right, and the three available signals disagree constantly on the open web.

The precedence here is deliberate and is the opposite of what is convenient:

  1. Magic bytes win. They describe the bytes that actually arrived.
  2. Extension is a weak hint. It is chosen by whoever named the file, is absent from most API URLs, and is trivially wrong.
  3. Content-Type is the weakest signal of the three, which surprises people. Servers routinely serve every archive as application/octet-stream, mislabel .tar.gz as application/x-gzip and vice versa, and — the case that matters — a captive portal or error page returns text/html with a 200 status, so a “download” completes and the saved file is a login page. Trusting the header there produces a file the user cannot open and cannot diagnose.

When the signals conflict, Detection::conflict says so, and the CLI warns. An HTML body delivered where an archive was expected is worth a warning even though nothing failed: it is the signature of an interception, and the byte count and status code both look fine.

§Sniffing is not decompression

Classification reads a prefix. It never decompresses, never rewrites, and never renames without being asked. A retriever that silently unpacked its output would be making a decision the caller did not delegate.

Structs§

Detection
The result of classifying an object.
Format
One recognised format.

Enums§

Category
Broad category, in the sense a download manager sorts by.
Evidence
Where a classification came from, in descending trustworthiness.

Functions§

catalogue
Every known format, for building a help screen, a GUI tooltip table, or a file-type filter.
describe
Label and explanation for a format name.
detect_format
Combine all three signals, with magic bytes taking precedence.
from_extension
Classify by filename or URL path extension.
from_magic
Classify by magic bytes alone.
from_media_type
Classify by a Content-Type header value.
known_extensions
Extensions this build recognises, for a GUI open/save filter.