Expand description
§content_scan
A pluggable, hierarchical content-scanning framework.
content_scan provides the building blocks for walking arbitrary
content (files, buffers, nested archives, embedded resources, etc.),
identifying its type, analyzing it, and recursively extracting inner
content. It is intentionally transport-agnostic: the caller supplies
anything that implements Content and the Scanner takes care of
dispatch, recursion, filtering, and result aggregation.
§Concepts
ContentType– a user-defined enum (typically derived with#[derive(ContentType)]) that enumerates the content kinds your application understands.Content– a trait describing a byte-addressable piece of content with aContentPathand a size. UseBufferContentfor in-memory data,FileContentfor a file on disk,FolderContentfor a directory, or implement it for your own sources.ContentPath– the path (or synthetic address) of a piece of content. UTF-8 strings and real OS paths, including non-UTF-8 names, are both representable; useContentPath::as_pathto open a file andContentPath::as_printable_stringto display it.ContentIdentifier– classifies a piece of content into aContentType(via magic bytes, file name, or extension) and validates the guess.ContentAnalyzer– inspects content and records information into theContext(global or per-object variable maps, plus a flat list ofFindings viaContext::add_finding). An analyzer can also queue extra extraction withContext::request_extract. Every analyzer implementsDependencies(typically via#[derive(Dependencies)]) so debug builds can check that required analyzers are registered with a lower priority. Findings may carry typedFindingMetadata; the default isNoMetadata.ContentExtractor– produces childContentitems from a parent (e.g. entries of an archive).create_sessionreceives anOwnedContentPtrto the parent and anExtractionContextnaming the region to look at, and returns anExtractionSession. The session then yields children viaadvance/extract. Extracted children are scanned recursively up to a configurable depth. Put per-session state on the session object, not on the extractor — one extractor instance is shared and sessions may nest.FolderExtractoris a ready-made implementation that walks a directory.Filter– optional inclusion/exclusion rules applied before any plugin runs on a piece of content.Scanner/ScannerBuilder– wires all plugins together and drives a scan. The scan produces aScanResultthat can be navigated as a tree ofScanContentHandles, and whosefindingsiterate every recordedFinding. UseScannerBuilder::with_metadatawhen findings need a customFindingMetadatatype.
§Example
ⓘ
use content_scan::*;
#[derive(Debug, Copy, Clone, Eq, PartialEq, Ord, PartialOrd, ContentType)]
#[repr(u16)]
enum MyTypes { Text }
struct TextId;
impl ContentIdentifier<MyTypes> for TextId {
fn identify_method(&self) -> Option<IdentifyMethod> {
Some(IdentifyMethod::Extension("txt"))
}
fn validate(&self, _: &mut dyn Content<MyTypes>) -> bool { true }
}
let mut scanner = ScannerBuilder::<MyTypes>::new()
.add_identifier(MyTypes::Text, TextId)
.build();
let mut content = BufferContent::<MyTypes>::new(b"hello", "hello.txt");
let result = scanner.scan(&mut content, true);
assert_eq!(result.objects_scanned(), 1);§Scanning a directory
ⓘ
use content_scan::*;
let mut scanner = ScannerBuilder::<MyTypes>::new()
.add_extractor(MyTypes::Folder, FolderExtractor::<MyTypes>::new(true, false))
.build();
let mut root = FolderContent::<MyTypes>::with_content_type("./src", MyTypes::Folder);
// `false`: the root folder itself is not tested against the filter
let result = scanner.scan(&mut root, false);Macros§
- var
- Expands a string literal to
varmap::Key::new(<fnv1a hash>).
Structs§
- Buffer
Content - An in-memory
Contentbacked by an owned byte buffer. - Content
Path - A path or address identifying a piece of content.
- Content
Reader - Sequential
Read+Seekadapter over anOwnedContentPtr. - Context
- Mutable state shared with plugins during a scan.
- Entry
- Metadata for a single entry produced by a
ContentExtractor. - Enum
VarMap - Heterogeneous map keyed by an enum implementing
EnumVarMapKey. - Enum
VarMap Pool - Pool of reusable
EnumVarMapvalues keyed byK. - Extract
Request Builder - Fluent builder for an extraction request, returned by
Context::request_extract. - Extraction
Context - Region of a parent
Contentthat aContentExtractorshould look at, plus optional analyzer-supplied parameters. - File
Content - A
Contentbacked by a file on disk. - Filter
- Compiled inclusion/exclusion policy applied to every content item before it is scanned.
- Filter
Builder - Builder for a
Filter. - Findigs
Iterator - Iterator over the findings of a
ScanResult. - Finding
- One finding recorded by an analyzer during a scan.
- Folder
Content - A
Contentstanding for a directory rather than a byte stream. - Folder
Extractor - A
ContentExtractorthat enumerates the entries of a directory. - Key
- Opaque key for
VarMap, holding a 64-bit hash. - NoMetadata
- Empty metadata type used when findings carry no extra data.
- Owned
Content Ptr - Non-owning handle to the parent
Contentof an extraction session. - Pool
Handle - Opaque generational handle to a map checked out from a pool.
- Readonly
- A
Syncread-only view of a map, safe to copy onto other threads. - Ready
Filter Builder - Finalized
FilterBuilderready to produce aFilter. - Scan
Content Handle - Opaque handle to a content object inside a
ScanResult. - Scan
Result - The outcome of a
Scanner::scancall. - Scanner
- The engine that drives a scan.
- Scanner
Builder - Fluent builder for
Scanner. - Shared
- A thread-safe, cheaply cloneable handle to a map (or any
T). - StrVar
Map - Heterogeneous map keyed by
&str. - StrVar
MapPool - Pool of reusable
StrVarMapvalues. - Value
- An encoded value and its associated arena borrow.
- Value
Builder - Helper for encoding
VarMapValuetypes into a map arena. - Value
Mut - Mutable view over a stored [
ValueKind] and its arena. - VarMap
- Heterogeneous map keyed by
Key. - VarMap
Pool - Pool of reusable
VarMapvalues. - ZipExtractor
- A
ContentExtractorthat unpacks the members of a ZIP archive. - ZipIdentifier
- A
ContentIdentifierfor ZIP archives.
Enums§
- Analysis
Outcome - Return value used by analyzers to steer the scan.
- Clear
Strategy - Strategy controlling which maps a pool retains when
clearis called. - Identify
Method - Fast, declarative way to identify a content type without running custom code.
- MemAlign
- Alignment bucket for data written into a map arena.
- Precedence
- Precedence of a
Filterrule.
Traits§
- Content
- A byte-addressable piece of content the scanner can operate on.
- Content
Analyzer - A plugin that inspects a piece of content and records information.
- Content
Extractor - A plugin that turns a container into a stream of child
Contentitems. - Content
Identifier - A plugin that classifies content into a
ContentType. - Content
Read Ext - Content
Type - Enumeration of content kinds understood by a scanner.
- Dependencies
- Debug-only name and required-analyzer list for a plugin.
- Enum
VarMap Key - Enum key type for
EnumVarMap. - Extraction
Session - A live extraction of children from one parent
Content. - Finding
Metadata - Typed extra data attached to a
Finding. - Scan
Observer - Callbacks for watching a scan as it runs.
- Stop
Condition - Predicate that can abort a scan early.
- VarMap
Value - Types that can be stored in and read from a varmap.
Derive Macros§
- Content
Type - Derives an implementation of
content_scan::ContentTypefor a#[repr(u16)]enum. - Dependencies
- Derives an implementation of
content_scan::Dependencies. - Enum
VarMap - Derives
EnumVarMapKeyfor a unit enum. - VarMap
Value - Derives
VarMapValuefor aCopystruct or enum.