Expand description
§file-identify
File identification for Rust: a faithful port of Python’s
identify, the library
pre-commit uses to decide which files a hook
runs on (types, types_or, exclude_types).
Given a path, it returns a set of tags such as
file, non-executable, python, text, derived from the file type, its
permissions, its name or extension, its shebang and its first kilobyte.
The lookup tables are generated from identify
IDENTIFY_VERSION — ALL_TAGS is exactly identify’s tag set.
§Quick start
use file_identify::{Tag, tags_from_filename, tags_from_path};
// Name only, no I/O.
let tags = tags_from_filename("Dockerfile.xenial");
assert_eq!(tags.to_string(), "dockerfile, text");
// Full identification (one lstat, at most one small read).
let tags = tags_from_path(&path)?;
assert!(tags.contains(Tag::FILE));
assert!(tags.contains("rust"));§Tags and tag sets
Tag is a 2-byte handle to one of the TAG_COUNT known tags;
TagSet is a Copy bitset with allocation-free set operations and
pre-commit’s filter in TagSet::matches_types. Parse user-supplied
names with Tag::from_name / TagSet::from_names; an unknown name is
exactly what pre-commit reports as Type tag '…' is not recognized.
§Identification API
| Function | identify equivalent | I/O |
|---|---|---|
tags_from_path, tags_from_path_with | tags_from_path | 1 lstat, ≤ 1 open + 1 KiB read |
tags_from_metadata | — (lstat metadata already at hand) | ≤ 1 open + 1 KiB read |
tags_from_meta | — (known Meta, e.g. from a git index mode) | ≤ 1 open |
classify() + Pending | — | none / caller-provided content |
tags_from_filename | tags_from_filename | none |
tags_from_interpreter | tags_from_interpreter | none |
file_is_text, is_text, is_text_bytes | file_is_text, is_text | ≤ 1 KiB |
parse_shebang_from_file, parse_shebang_from_file_with, parse_shebang, parse_shebang_bytes | parse_shebang_from_file, parse_shebang | first line(s) |
extensions and interpreters expose the raw tables.
§Differences from identify
Behaviour matches identify, including its quirks (Python shlex shebang
splitting, nix-shell shebangs, os.path.splitext rules, Windows
treating every file as executable). The deliberate exceptions keep the
library safe on hostile input:
- FIFOs and devices are never opened (identify blocks on a FIFO); they are
tagged
file+ mode + name tags, andbinaryif the name does not decide the encoding. - The shebang parser reads at most
MAX_SHEBANG_BYTES; a longer first line counts as “no shebang”. - On Unix,
os.access(X_OK)is evaluated from the file’s mode, owner and group (seeExecPolicy), ignoring ACLs andnoexecmounts.
Platform limitations (not deliberate):
- On Windows a directory junction (and any other name-surrogate reparse
point) is tagged
symlink: the standard library reports it as a symlink and does not expose the reparse tag. identify (os.lstat) reportsdirectoryfor a junction; true symlinks agree.
The git index fast path (Meta::from_git_mode, tags_from_meta)
describes the index, not the work tree. It matches identify only when
the two agree: in particular a 120000 entry is a symlink only if git
created one. With core.symlinks=false (the usual case on Windows) git
checks the link out as a regular file holding the target path, which
identify tags file, …, text. On Windows, or whenever
core.symlinks=false, do not trust 120000: lstat the file
(tags_from_path, tags_from_metadata) instead.
Re-exports§
pub use tags::IntoTag;pub use tags::Tag;pub use tags::TagSet;pub use tags::UnknownTagError;
Modules§
- extensions
- identify’s
extensionsmodule: the extension and file-name tables. - interpreters
- identify’s
interpretersmodule: the shebang interpreter table. - tags
- Tags, tag sets and identify’s tag-group constants.
Structs§
- File
Info - Pre-loaded file information for I/O-free identification (kept for
compatibility;
classify()+Pending::finish_bytesis the more flexible equivalent). - Meta
- What identification needs to know about a path besides its name: its
FileKindand whether it is executable. - Pending
- A classification waiting for file content (see
classify()). - TagTable
- One of identify’s lookup tables (
EXTENSIONS,NAMES, …): a static map from&strkeys toTagSets.
Enums§
- Classification
- Outcome of
classify(). - Exec
Policy - How to decide whether a file is executable.
- File
Kind - The kind of a file-system entry, as seen by
lstat(symlinks are not followed). - Identify
Error - Errors from identifying a path.
- Shlex
Error - Error from
shlex_split, mirroring theValueErrormessages of Python’sshlex.
Constants§
- IDENTIFY_
VERSION - Version of the Python identify release the tables in this crate were generated from.
- MAX_
SHEBANG_ BYTES - Upper bound on the bytes
parse_shebangconsumes (the#!line plus anynix-shellcontinuation lines). - TAG_
COUNT - Number of tags known to identify (
len(identify.ALL_TAGS)). - TEXT_
PROBE_ LEN - Number of leading bytes identify inspects to decide
textvsbinary.
Statics§
- ALL_
TAGS - Every tag known to identify (
identify.ALL_TAGS), sorted by name.
Functions§
- classify
- The I/O-free part of
identify.tags_from_path: everything that follows from the entry’sMetaand name. - file_
is_ text - Whether the file at
pathlooks like text, likeidentify.file_is_text(reads at mostTEXT_PROBE_LENbytes). - is_text
- Read up to
TEXT_PROBE_LENbytes fromreaderand report whether they look like text, exactly likeidentify.is_text. - is_
text_ bytes - Whether the first
TEXT_PROBE_LENbytes ofcontentlook like text, exactly likeidentify.is_text(io.BytesIO(content)). - parse_
shebang - Parse a shebang from the start of
reader, exactly likeidentify.parse_shebang, returning the command tokens (empty when there is no usable shebang). - parse_
shebang_ bytes parse_shebangfor in-memory content (infallible).- parse_
shebang_ from_ file - Parse the shebang of the file at
path, likeidentify.parse_shebang_from_file: empty unless the file is executable (ExecPolicy::PythonCompat: always on Windows), otherwiseparse_shebangon its content. - parse_
shebang_ from_ file_ with parse_shebang_from_filewith an explicitExecPolicydeciding whether the file is executable (and therefore read at all).- shlex_
split - Python’s
shlex.split(s)(POSIX mode,whitespace_split=True, no comment characters). - tags_
from_ filename - Identify a file from its name alone (no I/O), exactly like
identify.tags_from_filename. - tags_
from_ info - Identify a file from
FileInfowithout any I/O. - tags_
from_ interpreter - Tags for a shebang interpreter, exactly like
identify.tags_from_interpreter. - tags_
from_ meta - Identify a path whose
Metais already known (e.g. fromMeta::from_git_mode), withoutlstat. - tags_
from_ metadata - Identify a path whose
lstatmetadata (fs::symlink_metadata) the caller already holds, e.g. from an existence check (pre-commit filters files withos.path.lexistsbefore identifying them). - tags_
from_ path - Identify a path, exactly like
identify.tags_from_path, withExecPolicy::PythonCompat. - tags_
from_ path_ with tags_from_pathwith an explicitExecPolicy.
Type Aliases§
- Result
- Result type of the path-based functions.