Expand description
Scanner for uploaded files: images, SVG, PDF and PostScript.
A Scanner takes the bytes of an upload and what the uploader claimed
about it, and returns a Report: a Verdict and the Findings it
rests on. It checks that the file is the type it claims to be, that it
carries no active content, and that it is structurally sound. A file that
cannot be analyzed to the end is never reported clean.
use malmal::{ScanHints, Scanner, Verdict};
let scanner = Scanner::builder().build();
let hints = ScanHints { filename: Some("photo.jpg"), content_type: Some("image/jpeg") };
let report = scanner.scan_bytes(b"\x7fELF\x02\x01\x01", hints);
assert_eq!(report.verdict, Verdict::Malicious);
assert!(report.has("type.executable"));The type is told from the content; the file name and Content-Type in
ScanHints are claims to check, never to trust. Reject every file
whose verdict is not Verdict::Clean: a suspicious file is one the
scanner could not clear, not one it found harmless.
§What is checked
| Content | FileType | Looked for |
|---|---|---|
| JPEG, PNG and APNG, GIF, WebP, BMP, TIFF and BigTIFF, ICO, JPEG XL | Jpeg, Png, Gif, Webp, Bmp, Tiff, Ico, Jxl | broken structure, more pixels than Limits::max_pixels, code in metadata, data after the image, a second format in the same bytes |
| PSD and PSB, HEIC, AVIF | Psd, Heif, Avif | the same, of the container: the pixel data is not decoded |
| SVG | Svg | scripts, event handlers, javascript: and data: addresses, entities, embedded HTML, resources fetched from elsewhere |
| PDF, and Illustrator files from version 9 on | Pdf | JavaScript, actions that run by themselves, launches, PostScript, XFA forms, attachments (each scanned in turn), encryption, obfuscated names, renderer bombs, broken structure |
| PostScript, EPS, older Illustrator files | PostScript | file and device access, shell pipes, interpreter internals, code built at run time |
Every file is also checked for executables, server-side code (PHP, ASP,
JSP) and its SHA-256 against a HashList. Any other content is
type.unknown: office documents, archives and video are out of scope.
§Verdicts and policy
Each Finding has a code, a Severity and a message. The
Policy turns the worst severity into the verdict:
| Policy | Info | Low, Medium | High, Critical |
|---|---|---|---|
Policy::default | clean | suspicious | malicious |
Policy::strict | clean | suspicious (Low), malicious (Medium) | malicious |
Policy::allow lets one code through, at Info, for files that carry a
feature on purpose, such as attachments in e-invoices; the rest of the
file is still checked. Policy::with_override sets the severity of a
code, and Policy::with_allowed_types narrows the accepted types.
use malmal::{FileType, Limits, Policy, ScanHints, Scanner};
let policy = Policy::default()
.allow("pdf.embedded_file")?
.with_allowed_types([FileType::Jpeg, FileType::Png, FileType::Pdf]);
let limits = Limits {
max_file_size: 16 << 20,
..Limits::default()
};
let scanner = Scanner::builder()
.policy(policy)
.limits(limits)
.pdf_passwords(["statement password"])
.build();
let report = scanner.scan_bytes(b"<svg/>", ScanHints::default());
assert!(report.has("type.not_allowed"));§Finding codes
report::CODES lists every code. What each one means, its severity
and what to do about it are in
docs/findings.md.
A code keeps its meaning, and new codes arrive in minor releases.
Messages are for people and may be reworded at any time.
§Limits
Limits bound the file size, what compressed data may expand to, how
deep embedded files are followed, the pixels an image may declare and
the time a scan may take. What exceeds a limit is a limit.* finding,
so a file that was not analyzed to the end is never clean; limit.*
codes cannot be allowed. The work a scan does follows the size of the
file: a structure that would cost more is cut off with
limit.complexity.
The time limit is checked between steps, and a step cannot be stopped
from outside. Where a scan must never run on, scan in a process that can
be ended, as the malmald daemon of
malmal-cli does.
Scanner::scan_bytes takes the file whole. For a large file, map it
rather than read it, so that its pages stay the kernel’s to drop; what a
scan holds beyond the file is then what the structure costs, not the
size: a 1 GB PDF is scanned in less than 96 MB.
§Threads
A Scanner holds no per-scan state. Build one and share it between
threads, or clone it, which is cheap. A scan is CPU work: in an async
service, run it on a blocking thread.
§Stability
FileType, Report and Finding may grow, so match on them with
a wildcard.
§Features
cdr: rebuild raster images from their pixels withScanner::sanitize, so that nothing but the picture is kept. Off by default; it adds the codecs of theimagecrate to the build.cdr-svg: withcdr, also filter SVG files down to what draws a picture.cdr-pdf: withcdr, also render each page of a PDF and write a new PDF of nothing but the pictures of its pages.yara: match files against YARA rules withYaraRules. Off by default; it adds a WebAssembly runtime to the build.
The command-line scanner, the daemon (HTTP, a Unix socket, the ClamAV protocol and ICAP) and the C and Python bindings are in the repository.
Re-exports§
pub use detect::FileType;pub use hashlist::HashList;pub use hashlist::HashListError;pub use limits::Limits;pub use limits::MemoryLimit;pub use policy::AllowError;pub use policy::Policy;pub use report::Finding;pub use report::Report;pub use report::Severity;pub use report::Verdict;
Modules§
- cdr
cdr - Content disarm and reconstruction for raster images: the image is decoded to pixels and a new file is written from them. Whatever the scanner cannot see, the new file does not have: metadata, colour profiles, comments, unknown chunks, data after the image, a second format hiding in the same bytes.
- detect
- Telling a file’s type from its content. File names and
Content-Typeheaders are the uploader’s to choose, so they are only claims to check. - hashlist
- SHA-256 blocklists.
- limits
- Resource limits for a scan.
- policy
- Accepted file types and the mapping from findings to a verdict.
- report
- The result of a scan.
Structs§
- Scan
Hints - What the uploader claimed about the file. Never trusted, only checked.
- Scanner
- A configured scanner. It holds no per-scan state: build one and share it between threads, or clone it, which is cheap.
- Scanner
Builder - Builds a
Scanner; everything left unset keeps its default. - Yara
Error yara - A rule source that did not compile.
- Yara
Rules yara - A compiled rule set, cheap to clone and share between scanners.
Constants§
- MAX_
OVERDUE - The scans that
Scanner::scan_withinstopped waiting for and that are still running, at most, for a scanner and its clones together. Past it, files are refused at once withlimit.timeuntil one of them ends.