Skip to main content

Crate malmal

Crate malmal 

Source
Expand description

Scanner for uploaded files: images, SVG, PDF and PostScript.

A Scanner takes the bytes of an upload and what the uploader claimed about it, and returns a Report: a Verdict and the Findings it rests on. It checks that the file is the type it claims to be, that it carries no active content, and that it is structurally sound. A file that cannot be analyzed to the end is never reported clean.

use malmal::{ScanHints, Scanner, Verdict};

let scanner = Scanner::builder().build();
let hints = ScanHints { filename: Some("photo.jpg"), content_type: Some("image/jpeg") };
let report = scanner.scan_bytes(b"\x7fELF\x02\x01\x01", hints);
assert_eq!(report.verdict, Verdict::Malicious);
assert!(report.has("type.executable"));

The type is told from the content; the file name and Content-Type in ScanHints are claims to check, never to trust. Reject every file whose verdict is not Verdict::Clean: a suspicious file is one the scanner could not clear, not one it found harmless.

§What is checked

ContentFileTypeLooked for
JPEG, PNG and APNG, GIF, WebP, BMP, TIFF and BigTIFF, ICO, JPEG XLJpeg, Png, Gif, Webp, Bmp, Tiff, Ico, Jxlbroken structure, more pixels than Limits::max_pixels, code in metadata, data after the image, a second format in the same bytes
PSD and PSB, HEIC, AVIFPsd, Heif, Avifthe same, of the container: the pixel data is not decoded
SVGSvgscripts, event handlers, javascript: and data: addresses, entities, embedded HTML, resources fetched from elsewhere
PDF, and Illustrator files from version 9 onPdfJavaScript, actions that run by themselves, launches, PostScript, XFA forms, attachments (each scanned in turn), encryption, obfuscated names, renderer bombs, broken structure
PostScript, EPS, older Illustrator filesPostScriptfile and device access, shell pipes, interpreter internals, code built at run time

Every file is also checked for executables, server-side code (PHP, ASP, JSP) and its SHA-256 against a HashList. Any other content is type.unknown: office documents, archives and video are out of scope.

§Verdicts and policy

Each Finding has a code, a Severity and a message. The Policy turns the worst severity into the verdict:

PolicyInfoLow, MediumHigh, Critical
Policy::defaultcleansuspiciousmalicious
Policy::strictcleansuspicious (Low), malicious (Medium)malicious

Policy::allow lets one code through, at Info, for files that carry a feature on purpose, such as attachments in e-invoices; the rest of the file is still checked. Policy::with_override sets the severity of a code, and Policy::with_allowed_types narrows the accepted types.

use malmal::{FileType, Limits, Policy, ScanHints, Scanner};

let policy = Policy::default()
    .allow("pdf.embedded_file")?
    .with_allowed_types([FileType::Jpeg, FileType::Png, FileType::Pdf]);
let limits = Limits {
    max_file_size: 16 << 20,
    ..Limits::default()
};
let scanner = Scanner::builder()
    .policy(policy)
    .limits(limits)
    .pdf_passwords(["statement password"])
    .build();
let report = scanner.scan_bytes(b"<svg/>", ScanHints::default());
assert!(report.has("type.not_allowed"));

§Finding codes

report::CODES lists every code. What each one means, its severity and what to do about it are in docs/findings.md. A code keeps its meaning, and new codes arrive in minor releases. Messages are for people and may be reworded at any time.

§Limits

Limits bound the file size, what compressed data may expand to, how deep embedded files are followed, the pixels an image may declare and the time a scan may take. What exceeds a limit is a limit.* finding, so a file that was not analyzed to the end is never clean; limit.* codes cannot be allowed. The work a scan does follows the size of the file: a structure that would cost more is cut off with limit.complexity.

The time limit is checked between steps, and a step cannot be stopped from outside. Where a scan must never run on, scan in a process that can be ended, as the malmald daemon of malmal-cli does.

Scanner::scan_bytes takes the file whole. For a large file, map it rather than read it, so that its pages stay the kernel’s to drop; what a scan holds beyond the file is then what the structure costs, not the size: a 1 GB PDF is scanned in less than 96 MB.

§Threads

A Scanner holds no per-scan state. Build one and share it between threads, or clone it, which is cheap. A scan is CPU work: in an async service, run it on a blocking thread.

§Stability

FileType, Report and Finding may grow, so match on them with a wildcard.

§Features

  • cdr: rebuild raster images from their pixels with Scanner::sanitize, so that nothing but the picture is kept. Off by default; it adds the codecs of the image crate to the build.
  • cdr-svg: with cdr, also filter SVG files down to what draws a picture.
  • cdr-pdf: with cdr, also render each page of a PDF and write a new PDF of nothing but the pictures of its pages.
  • yara: match files against YARA rules with YaraRules. Off by default; it adds a WebAssembly runtime to the build.

The command-line scanner, the daemon (HTTP, a Unix socket, the ClamAV protocol and ICAP) and the C and Python bindings are in the repository.

Re-exports§

pub use detect::FileType;
pub use hashlist::HashList;
pub use hashlist::HashListError;
pub use limits::Limits;
pub use limits::MemoryLimit;
pub use policy::AllowError;
pub use policy::Policy;
pub use report::Finding;
pub use report::Report;
pub use report::Severity;
pub use report::Verdict;

Modules§

cdrcdr
Content disarm and reconstruction for raster images: the image is decoded to pixels and a new file is written from them. Whatever the scanner cannot see, the new file does not have: metadata, colour profiles, comments, unknown chunks, data after the image, a second format hiding in the same bytes.
detect
Telling a file’s type from its content. File names and Content-Type headers are the uploader’s to choose, so they are only claims to check.
hashlist
SHA-256 blocklists.
limits
Resource limits for a scan.
policy
Accepted file types and the mapping from findings to a verdict.
report
The result of a scan.

Structs§

ScanHints
What the uploader claimed about the file. Never trusted, only checked.
Scanner
A configured scanner. It holds no per-scan state: build one and share it between threads, or clone it, which is cheap.
ScannerBuilder
Builds a Scanner; everything left unset keeps its default.
YaraErroryara
A rule source that did not compile.
YaraRulesyara
A compiled rule set, cheap to clone and share between scanners.

Constants§

MAX_OVERDUE
The scans that Scanner::scan_within stopped waiting for and that are still running, at most, for a scanner and its clones together. Past it, files are refused at once with limit.time until one of them ends.