1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
//! Shared read-side plumbing for a capped, deadline-bounded HTTP body
//! read, used by both the library's document-triggered image fetch
//! (`net_guard::fetch_url`) and the CLI's operator-typed `--url`
//! fetch (`src/bin/main.rs`).
//!
//! The binary is a separate crate from this library and can only see
//! its public API, and none of this is API we want to publish just
//! for internal plumbing — so, mirroring the pattern the test suite
//! already uses for `tests/render/common.rs`
//! (`#[path = "render/common.rs"] mod common;`), `src/bin/main.rs`
//! includes this exact file via
//! `#[path = "../lib/render/net_read.rs"] mod net_read;` and compiles
//! it a second time as part of the binary crate. `net_guard.rs` isn't
//! shared the same way: it also carries the SSRF host-block predicates
//! (`ipv4_blocked` and friends), which the CLI's operator-typed fetch
//! deliberately does NOT apply (see the comment at that call site in
//! `main.rs`) — including the whole file would either pull in that
//! guard by accident or leave those items unused-and-warning under
//! `-D warnings` in the binary. This file carries only the read-side
//! plumbing both call sites genuinely share.
use Read;
use Instant;
/// The hard byte ceiling on a fetched response body. Shared by the
/// library's document-triggered image fetch and the CLI's
/// operator-typed `--url` fetch so the two can't quietly drift apart
/// — they used to each hardcode `10 * 1024 * 1024` separately.
pub const MAX_FETCH_BYTES: u64 = 10 * 1024 * 1024;
/// `Read` adapter enforcing a hard wall-clock deadline across every
/// read call, on top of (not instead of) the client's own idle
/// timeout. reqwest's blocking `.timeout()` behaves as an IDLE
/// timeout while reading a response body — any forward progress
/// resets it — so a server trickling data slower than the size cap
/// but faster than the idle timeout can keep a `read_to_end` alive
/// indefinitely. This wrapper fails the read once wall-clock time
/// passes `deadline`, bounding total transfer time to roughly the
/// deadline plus one more idle-timeout period: the read straddling
/// the deadline can still block up to the idle timeout before this
/// adapter gets a chance to check the clock again. Bounded and
/// honest, not an instantaneous cutoff.
pub
/// Read `reader` to completion under a [`DeadlineReader`] wall-clock
/// cutoff, capped one byte past `MAX_FETCH_BYTES` via `Read::take` so
/// an over-size (or `Content-Length`-lying) body is detectable without
/// ever buffering the whole thing. Returns the raw bytes — which may
/// be `MAX_FETCH_BYTES + 1` bytes long — leaving it to the caller to
/// size-check and phrase its own "too big" error message (the
/// library's and the CLI's read the same way but word the error
/// differently).
pub