Skip to main content

Module verify

Module verify 

Source
Expand description

Read-time integrity verification for streaming object bodies.

Object-store sources (S3, GCS) read an object body through an async reader and trust the SDK to either deliver the whole body or surface an error. A transfer that terminates early but cleanly (a truncated stream that still yields EOF) would otherwise be parsed and emitted as a complete object — silent data loss with no error (#161).

VerifyingReader wraps the raw body reader (below any decompression layer, so byte counts and checksums cover the stored bytes, not the decoded ones) and runs a set of IntegrityChecks once the underlying reader reaches EOF. A failed check surfaces as an io::Error of kind InvalidData on the final read, which the connector maps to FaucetError::Source.

The built-in LengthCheck guards against truncation. Connector crates implement IntegrityCheck over their own hash libraries for opt-in checksum verification.

Structs§

LengthCheck
Built-in IntegrityCheck that fails when the number of bytes read does not match the length advertised by the object store (Content-Length for S3, object size for GCS). Catches a cleanly-truncated transfer that would otherwise be accepted as a complete object.
VerifyingReader
An AsyncRead adapter that observes every byte read from inner and runs the configured IntegrityChecks at EOF. Wrap the raw network reader before any BufReader / decompression layer.

Traits§

IntegrityCheck
A check fed the raw bytes of an object body as they stream past, and asked to validate once the underlying reader reaches EOF.