content_inspector
A simple library for fast inspection of binary buffers to guess the type of content.
This is mainly intended to quickly determine whether a given buffer contains "binary"
or "text" data. Programs like grep
or git diff
use similar mechanisms to decide whether
to treat some files as "binary data" or not.
The analysis is based on a very simple heuristic: Searching for NULL bytes (indicating "binary" content) and the detection of special byte order marks (indicating a particular kind of textual encoding). Note that this analysis can fail. For example, even if unlikely, UTF-8-encoded text can legally contain NULL bytes. Conversely, some particular binary formats (like binary PGM) may not contain NULL bytes. Also, for performance reasons, only the first 1024 bytes are checked for the NULL-byte (if no BOM was detected).
Usage
use ;
assert_eq!;
assert_eq!;
assert!;
CLI example
This crate also comes with a small example command-line program (see examples/inspect.rs
) that demonstrates the usage:
> inspect
> inspect
If you only want to detect whether something is a binary or text file, this is about a factor of 250 faster than file --mime ...
.