1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
//! Progressive streaming: the emitter/tracer split described in `serve`'s module
//! docs.
//!
//! This module holds every piece that's testable as pure logic, independent of any
//! actual socket: [`PendingDelta`] (delta coalescing), [`next_batch_size`] (adaptive
//! sub-batch sizing), and [`downsample_preview`] (the cumulative reduced-resolution
//! snapshot `PREVIEW` sends). [`run_stream`] wires these together with a tracer thread
//! and drives the actual `Read + Write` stream; see its own doc comment and `serve`'s
//! module docs for the full architecture.
use ;
/// The viewer's own share of a `FinalImageRequest` (v16): [`ContributionSlot`] and its
/// `await_until`/`receive` rendezvous -- see this module's own doc comment.
pub use ;
pub use ;
pub use ;
pub use trace_range;
/// What [`run_stream_with`] streams: the request (its id, scene, range and
/// `StreamConfig`), the connection's negotiated payload encoding, what the request
/// turns into at the end ([`Output`]), and (v16) the viewer's own reserved share of a
/// final picture, if any.
/// The stall window a directly served request gets: `StreamSpec::stall_timeout` for a
/// plain worker and a coordinator's own-lane route.
///
/// Generous on purpose. The first sample lands only after the tracer's calibration probe
/// and, on a cold GPU, a pipeline compile; a huge export sub-batch is sized to about a
/// second of work. Two minutes is far past either, yet short enough that a wedged lane
/// stops holding a batch client's slot for good.
pub const PRODUCER_STALL_TIMEOUT: Duration = from_secs;
/// This worker's cadence FLOOR, advertised in `WELCOME::min_cadence_ms` -- see that
/// field's doc comment. Matches [`TARGET_SUBBATCH`], the sub-batch duration
/// [`next_batch_size`] targets: emitting faster than one sub-batch completes is not
/// meaningful, since there is nothing new to send in between.
pub const MIN_CADENCE_FLOOR_MS: u32 = 100;
/// How long `emitter::poll_for_client_message` and `crate::serve::tilt::poll_for_cancel`
/// wait for the REMAINDER of a message once its first byte has arrived, instead of
/// blocking forever. Shared here so the two independent implementations of this pattern
/// can't drift to different bounds.
///
/// Both read a message's length-prefix first byte with a raw, timeout-tolerant `read()`
/// so a timeout can only land BEFORE any byte has arrived. Once a byte arrives, the
/// remainder gets this bounded window instead of `None`, so a peer that sends a partial
/// frame and then goes silent hits a protocol error (tearing down the connection)
/// instead of hanging the reading thread forever.
///
/// 5s is generous relative to both functions' millisecond-scale first-byte poll
/// interval: a real message (tiny -- a `CANCEL` or pipelined `RenderRequest`) completes
/// well under a second even on a slow link, leaving slack for congestion without
/// leaving a stalled connection blocking indefinitely.
pub const FRAME_REMAINDER_TIMEOUT: Duration = from_secs;
/// The hard, client-`cadence_ms`-independent ceiling on how long [`run_stream`]'s
/// emitter may go without writing some `StreamEvent` (a bare `Progress` heartbeat, if
/// nothing else was due) while a request is streaming.
///
/// The editor's remote-render connection declares a worker dead after its own
/// `LIVENESS_TIMEOUT` (8s) with no `StreamEvent` received (30s grace before the first
/// event); this is a quarter of that, leaving slack for one lost heartbeat without the
/// next also missing the deadline.
///
/// Can't just be `request.stream.cadence_ms`: that's client-chosen with only a floor
/// ([`MIN_CADENCE_FLOOR_MS`]), and a large export legitimately requests a large cadence
/// for fewer, bigger payloads that could otherwise go silent past `LIVENESS_TIMEOUT`.
/// The emitter loop wakes at least this often unconditionally and heartbeats with the
/// latest `samples_done` whenever nothing else went out; a due cadence tick is never
/// delayed by this.
pub const HEARTBEAT_INTERVAL: Duration = from_secs;
/// True if `e` is what a stream-level timeout looks like on this crate's two real
/// transports (a raw `TcpStream`, and TLS via `rustls::StreamOwned`) -- the single place
/// both `emitter::poll_for_client_message` and `crate::serve::tilt::poll_for_cancel`
/// decide "nothing pending yet" from a genuine protocol error.
///
/// A raw socket's timeout elapsing surfaces as [`std::io::ErrorKind::WouldBlock`] or
/// [`std::io::ErrorKind::TimedOut`] depending on platform.
/// [`std::io::ErrorKind::WriteZero`] is a third, non-obvious member once TLS is
/// involved: `rustls`'s `Stream::read`/`write` drive `complete_io()` internally, and a
/// single call can require both a socket read and write under the hood. When that
/// inner I/O times out, rustls reports `WriteZero` instead of propagating the original
/// `WouldBlock`/`TimedOut` -- without treating it the same, a merely-slow-but-alive
/// peer's poll could be misclassified as a fatal protocol error.
/// A stream that can have a short read timeout applied, so [`run_stream`]'s emitter loop
/// can poll for an incoming `CANCEL` without ever blocking longer than the timeout.
///
/// Implemented for the real transports (`TcpStream`, and `rustls::StreamOwned` wrapping
/// one) by delegating to the underlying socket. A test double can implement this to
/// toggle its own "exhausted input means `WouldBlock`, not EOF" behavior -- see
/// `serve`'s `tests::DuplexHalf`.
/// Mirrors `std::io::Read`/`Write`'s blanket impls for `&mut T`, so a test can drive
/// `run_stream` through a `&mut SomeTestDouble` and still inspect it afterward.
/// Caches the last read-timeout value applied to a stream through
/// [`TimeoutCache::apply`], so a caller that (like [`emitter::run_stream`]'s loop) asks
/// for the same value on every poll only pays the underlying `set_read_timeout`
/// syscall once, not every time.
///
/// # Why this exists
///
/// `run_stream`'s emitter calls [`emitter::poll_for_client_message`] every
/// [`emitter::EMITTER_POLL`] (20ms), which calls `set_read_timeout` up to twice per
/// call (once for the poll window, once more after the first byte of a message
/// arrives). On an idle streaming connection (no `CANCEL`, no pipelined
/// `RenderRequest` ever arriving) that would be ~100 setsockopt syscalls/s for no
/// behavioral benefit, since the value being set is identical to what's already on the
/// socket. [`Self::apply`] makes the second call a no-op whenever the requested value
/// hasn't changed since the last one this cache actually applied.
///
/// Owned by `run_stream` for the life of one request and passed by `&mut` into
/// `poll_for_client_message`; a fresh cache per request means a new request never
/// trusts an assumption inherited from a previous one about what's currently on the
/// socket -- see [`Self::new`].
/// Whether [`TimeoutCache`] has applied a read timeout yet, and if so, the last value
/// it applied -- a named alternative to `Option<Option<Duration>>` (flagged by
/// `clippy::option_option`, and less self-explanatory at every call site than a type
/// whose variants say what they mean).
/// Remembers the last socket timeout applied so repeat settings skip the syscall.
/// A stream that can have a write timeout applied, so [`run_stream`]'s emitter can never
/// block on a single `write()` call for longer than [`emitter::WRITE_TIMEOUT`] --
/// confirmed root cause of two real-world reports (the worker appearing to "stop sending
/// data" mid-render, and a `CANCEL` not taking effect promptly). [`run_stream`]'s
/// emitter both writes payloads and polls for `CANCEL` sequentially in one loop;
/// without a write timeout, a peer whose read loop falls behind blocks that thread
/// indefinitely. A bounded timeout turns that into "blocks for at most one timeout,
/// then this connection ends" -- the only safe option, since a `write_all` that has
/// already pushed part of a length-prefixed frame cannot be abandoned mid-message
/// without desyncing the peer's framing. Implemented the same way as [`TimeoutRead`].
/// How [`run_stream`] ended, for `handle_connection`'s caller to decide what (if
/// anything) still needs to be written. `run_stream` has already sent every
/// `StreamEvent` for [`Completed`](StreamOutcome::Completed) (normal finish or
/// cancellation both produce a `DONE`, just with a different `cancelled` flag); only
/// [`TracePanicked`](StreamOutcome::TracePanicked) requires the caller to send
/// `StreamEvent::Error` itself. Only half of what [`run_stream`] returns -- see its own
/// doc comment for the `Option<RenderRequest>` alongside it.
/// The short label the per-request `info!` lines use: `completed`, `trace panicked` or
/// `failed code N`.