protocol_droid/adapters/primary/transport/http/stream.rs
1//! Turning a [`Resume`] into a live `text/event-stream` body.
2//!
3//! # What this is for
4//!
5//! [`resume`] decides *what* a client that sent `Last-Event-ID` should get, and
6//! [`sse`] encodes one frame. Neither writes to a socket, and the gap between a
7//! decision and a byte on the wire is where resumability is either real or
8//! decorative. This module is that gap: it owns the response, the header set a
9//! proxy needs in order not to buffer it, and the loop that writes the buffered
10//! frames and then stays open.
11//!
12//! # The live tail is a cursor, not a queue
13//!
14//! This is the whole design, and it is worth stating plainly because the
15//! obvious alternative is wrong in a way that only shows up in production.
16//!
17//! The obvious shape is: take the `Resume`, subscribe to a channel, write the
18//! buffered frames, then forward the channel. That has a race in it. Between
19//! the moment the ring was read to build the `Resume` and the moment the
20//! subscription is live, a push can land — and a frame handed to a channel
21//! nobody is receiving from yet is a frame that is gone. The client was told it
22//! was resuming from cursor 40, it next reconnects with 43, and 41 is silently
23//! absent. Nothing errors. Nothing logs. Resumability has become decorative,
24//! and the test that would catch it needs a real scheduler and a frame pushed
25//! at exactly the wrong moment.
26//!
27//! So there is no subscription. The stream holds a **cursor** — the
28//! [`EventId`] it has written up to — and every time it is polled it asks the
29//! ring for everything after that cursor. [`EventTail::plan`] returns the
30//! `Resume` and the watermark it was computed at *under one lock*, so the
31//! cursor the stream starts from is exactly the one the decision was made
32//! against, and a frame pushed one instruction later has an id strictly greater
33//! than it. The first poll returns it. There is no interval in which a frame
34//! can be missed, because "read the ring" and "start receiving" are not two
35//! steps — they are the same operation, performed again.
36//!
37//! # Why the stream re-uses `decide` rather than inventing a second answer
38//!
39//! Every decision the live tail makes goes through [`decide`], the same
40//! function that answers a client which has just reconnected. That is not
41//! tidiness, it is the property that makes a slow client safe: a client reading
42//! slowly enough for the ring to evict past its cursor gets a `Gap`, and the
43//! writer emits the gap notice — the identical frame, built by the identical
44//! code, carrying no `id:` — rather than skipping the evicted events in
45//! silence. A separate code path for "the stream fell behind" would be a
46//! second opinion about what a gap means, and the two would disagree exactly
47//! when it mattered.
48//!
49//! Note what a mid-stream gap does to the cursor: it leaves it at
50//! `evicted_through`, not at `last`. The events the ring still holds past the
51//! eviction point were never lost, so the next poll delivers them. A client
52//! that fell behind is told what it missed *and* handed everything it can still
53//! have, which is the same answer a reconnect would produce.
54//!
55//! Two of `decide`'s arms have no business appearing mid-stream and are handled
56//! rather than ignored. A `Refused(UnknownCursor)` means the ring was
57//! [`clear`](crate::domain::EventRing::clear)ed or the cursor is from a retired
58//! stream; there is no continuation of it, so the stream **closes** rather than
59//! pretending. A `Refused(Handler)` means a payload had no JSON form — the same
60//! server-side bug a reconnect is refused for — and the stream closes for the
61//! same reason.
62//!
63//! # The lock order, and why it is the whole cancellation story
64//!
65//! A dropped client must release its subscription, or the ring accumulates
66//! wakers for futures that no longer exist. The registration is therefore
67//! removed in [`Drop`], keyed by a **unique registration id** and not by
68//! [`Waker::will_wake`]. Two streams served to the same task carry the same
69//! waker, and a `will_wake` match would let one stream's drop unregister the
70//! other's — silently freezing a live client. The id makes that
71//! unrepresentable.
72//!
73//! Two mutexes are held here and the order is fixed: **ring before waiters**,
74//! always. Registration happens *while the ring lock is still held*, which is
75//! what closes the wake race: a stream that found nothing and registered its
76//! waker did both in the same critical section as the read that found nothing,
77//! so a concurrent `push` either completed before the read — and the frame is
78//! there — or waits for the lock and then finds the waker already registered.
79//! There is no third interleaving. Releasing the ring lock before registering
80//! would reopen exactly the window the design above exists to close.
81//!
82//! # Why nothing here touches the session engine
83//!
84//! [`SessionEngine::dispatch`](crate::application::SessionEngine::dispatch)
85//! takes `&mut self` for the whole of a handler, so a body stream that tried to
86//! drive the engine would be asking for a mutable borrow of something the
87//! handler already holds. That is a deadlock, and a silent one, because the
88//! stream is parked rather than running — it would hang rather than fail. This
89//! module never mentions the engine. The engine stays in the request task that
90//! owns it; anything the engine produces is published into the shared
91//! [`EventTail`], which is `Send + Sync` and interior-mutable, and the body
92//! stream only ever reads from it. Stage 2d-ii wires those two together; this
93//! module is deliberately the half that does not care which.
94//!
95//! # Protocol neutrality
96//!
97//! There is no MCP method name, no revision type, and no version check in this
98//! file. The payload is a `T: FramePayload` supplied by the caller, the
99//! framing is the transport's, and the decision comes from the protocol-neutral
100//! ring. A2A and ACP bind to the same [`EventTail`] with a different payload
101//! and a different decision function, and need nothing here rewritten.
102//!
103//! # Example
104//!
105//! ```
106//! use core::num::NonZeroUsize;
107//! use serde_json::{Value, json};
108//! use protocol_droid::adapters::primary::transport::http::EventTail;
109//! use protocol_droid::domain::EventId;
110//!
111//! // A tail is a ring plus a way to wake whoever is reading it.
112//! let tail = EventTail::<Value>::new(NonZeroUsize::new(8).ok_or("capacity")?);
113//! tail.push(json!({ "hello": "world" }));
114//!
115//! // A client quoting `Last-Event-ID: 1` is already current: silence, then the
116//! // live stream. No frame, no close, and no cursor to move.
117//! let planned = tail.plan(Some("1"));
118//! assert!(planned.resume().pending().is_empty());
119//! assert_eq!(planned.cursor(), EventId::new(1));
120//!
121//! // A push after the decision does not rewrite it — the decision is a snapshot
122//! // of one moment, and a client that re-reads it would be told a different
123//! // story each time. What makes the new event reach the client is the cursor:
124//! // the stream re-reads from it on every poll, so a push that lands after
125//! // `plan` is still delivered.
126//! tail.push(json!({ "later": true }));
127//! assert_eq!(planned.resume().last(), Some(EventId::new(1)));
128//! # Ok::<(), Box<dyn std::error::Error>>(())
129//! ```
130
131use core::num::NonZeroUsize;
132use core::pin::Pin;
133use core::sync::atomic::{AtomicU64, Ordering};
134use core::task::{Context, Poll};
135use core::time::Duration;
136use std::collections::VecDeque;
137use std::sync::{Arc, Mutex, MutexGuard, PoisonError};
138use std::task::Waker;
139
140use axum::body::{Body, Bytes};
141use axum::http::{StatusCode, header};
142use axum::response::{IntoResponse, Response};
143use futures_core::Stream;
144use thiserror::Error;
145
146use super::frame_payload::FramePayload;
147use super::resume::{Resume, decide, plan};
148use super::sse::SseEvent;
149use crate::domain::{EventId, EventRing};
150
151/// The media type a resumable stream is served as.
152///
153/// A `const` rather than a literal at the call site because it is part of the
154/// wire contract, and a value written twice is a value that will one day be
155/// written twice differently.
156const EVENT_STREAM: &str = "text/event-stream";
157
158/// The header that tells nginx not to buffer the response.
159///
160/// Without it nginx will hold a `text/event-stream` in its proxy buffer and
161/// deliver it in multi-kilobyte lumps, which turns a live stream into one that
162/// appears dead and then delivers everything at once on a full buffer. This is
163/// the commonest way a working SSE server looks broken in production, and it is
164/// completely invisible from the code that emits the frames.
165const ACCEL_BUFFERING: &str = "x-accel-buffering";
166
167/// The keep-alive: an SSE comment line, which dispatches nothing.
168///
169/// A comment, and not an empty event and not a close. A close is a lie — the
170/// stream is healthy. An empty event is worse than a lie: a client that
171/// dispatches it receives a message with no payload and no `id:`, and a client
172/// that treats an unrecognised message as a protocol event has been handed
173/// something the specification never described. The event-stream standard
174/// defines a line beginning with `:` as a comment for exactly this purpose, and
175/// every conforming reader ignores it.
176///
177/// A `const` and not an [`SseEvent`] because [`SseEvent`] encodes *data* frames
178/// and has no comment form — a comment carries no data, and widening
179/// `SseEvent` to represent one would put a variant on a type whose `encode`
180/// output every existing test in `sse` would have to be re-read against.
181pub const KEEP_ALIVE_FRAME: &str = ":ping\n\n";
182
183/// Why a body writer is fallible.
184///
185/// The only reachable variant is the response assembly. A body whose frames are
186/// already encoded cannot fail mid-stream, and the [`Stream`] item type is a
187/// `Result` only because [`Body::from_stream`] requires one — an unencodable
188/// payload is turned into a refusal by [`decide`] long before it could reach
189/// the writer.
190#[derive(Debug, Clone, PartialEq, Eq, Error)]
191pub enum StreamError {
192 /// The response could not be assembled.
193 ///
194 /// Every header this module sets is a compile-time literal, so nothing in
195 /// the crate can actually trigger this today. It is handled rather than
196 /// unwrapped because the alternative is a panic on a request path, and a
197 /// request path is the worst place to put one.
198 #[error("the event-stream response could not be assembled: {0}")]
199 Response(String),
200}
201
202impl From<axum::http::Error> for StreamError {
203 fn from(error: axum::http::Error) -> Self {
204 Self::Response(error.to_string())
205 }
206}
207
208/// The wakers of every stream currently parked on a tail, each with the id that
209/// makes it individually removable.
210///
211/// A `Vec` rather than a set: streams are one per connected client and
212/// registration is a `push`, so the linear scan on the removal path is cheaper
213/// than a hash. The *ids* are what make removal correct; the shape is only the
214/// cheapest thing that carries them.
215#[derive(Debug, Default)]
216struct Waiters {
217 entries: Vec<(u64, Waker)>,
218}
219
220/// State shared between a publisher and every stream reading from it.
221///
222/// Two mutexes, always locked ring-then-waiters. See the module docs for why
223/// that order is load-bearing and why registration happens under the ring lock.
224#[derive(Debug)]
225struct Shared<T> {
226 ring: Mutex<EventRing<T>>,
227 waiters: Mutex<Waiters>,
228 next_registration: AtomicU64,
229}
230
231impl<T> Shared<T> {
232 /// The ring, recovering from poisoning rather than propagating it.
233 ///
234 /// A poisoned mutex means some thread panicked while holding it. What is
235 /// behind it is a `Vec` of constructed events and a `Vec` of wakers, and
236 /// neither can be left half-written: a push is a `Vec::push` of a value that
237 /// was already built. Propagating the poison would turn one panicking
238 /// handler into a permanently dead endpoint, which is a far worse failure
239 /// than continuing from a ring that is still structurally sound.
240 fn ring(&self) -> MutexGuard<'_, EventRing<T>> {
241 self.ring.lock().unwrap_or_else(PoisonError::into_inner)
242 }
243
244 /// The waiter list, recovering from poisoning on the same reasoning.
245 fn waiters(&self) -> MutexGuard<'_, Waiters> {
246 self.waiters.lock().unwrap_or_else(PoisonError::into_inner)
247 }
248
249 /// A registration id no other live registration holds.
250 fn next_registration(&self) -> u64 {
251 // A relaxed fetch-add is right for an id that only has to be unique.
252 // Nothing is published through it, and a collision would need two
253 // threads to receive the same value, which the atomic add prevents
254 // regardless of ordering.
255 self.next_registration.fetch_add(1, Ordering::Relaxed)
256 }
257}
258
259/// A shared, wakeable, replayable source of stream events.
260///
261/// What a session publishes into and what a response body reads from. It is
262/// `Send + Sync` and holds no borrow of anything, which is the property that
263/// lets a body stream outlive the request task that built it — see the module
264/// docs on why the stream must not be the thing holding the engine.
265///
266/// Cloning is cheap and shares: every clone reads and writes the same ring, so
267/// a router can keep one in its state and hand a reference to each response.
268///
269/// # Example
270///
271/// ```
272/// use core::num::NonZeroUsize;
273/// use serde_json::{Value, json};
274/// use protocol_droid::adapters::primary::transport::http::EventTail;
275///
276/// let capacity = NonZeroUsize::new(4).ok_or("capacity")?;
277/// let tail = EventTail::<Value>::new(capacity);
278/// for index in 0..3 {
279/// tail.push(json!({ "index": index }));
280/// }
281/// // A client that has seen all three is current and gets no frames.
282/// assert!(tail.plan(Some("3")).resume().pending().is_empty());
283/// # Ok::<(), Box<dyn std::error::Error>>(())
284/// ```
285#[derive(Debug)]
286pub struct EventTail<T> {
287 shared: Arc<Shared<T>>,
288}
289
290impl<T> EventTail<T> {
291 /// A tail over a ring retaining `capacity` events.
292 ///
293 /// [`RECOMMENDED_CAPACITY`](crate::domain::RECOMMENDED_CAPACITY) is the
294 /// right value for a caller that has not measured its payloads; pass a real
295 /// count for one that has.
296 #[must_use]
297 pub fn new(capacity: NonZeroUsize) -> Self {
298 Self {
299 shared: Arc::new(Shared {
300 ring: Mutex::new(EventRing::new(capacity)),
301 waiters: Mutex::new(Waiters::default()),
302 next_registration: AtomicU64::new(0),
303 }),
304 }
305 }
306
307 /// How many events are currently retained.
308 ///
309 /// For a log line or a test assertion. Taking the ring lock is why this is
310 /// not on any per-frame path.
311 #[must_use]
312 pub fn len(&self) -> usize {
313 self.shared.ring().len()
314 }
315
316 /// Whether nothing is retained.
317 #[must_use]
318 pub fn is_empty(&self) -> bool {
319 self.shared.ring().is_empty()
320 }
321
322 /// The newest event id ever issued, or the watermark if nothing has been.
323 #[must_use]
324 pub fn last(&self) -> EventId {
325 self.shared.ring().last()
326 }
327}
328
329impl<T> Clone for EventTail<T> {
330 fn clone(&self) -> Self {
331 Self {
332 shared: Arc::clone(&self.shared),
333 }
334 }
335}
336
337impl<T> Default for EventTail<T> {
338 fn default() -> Self {
339 Self::new(crate::domain::RECOMMENDED_CAPACITY.into())
340 }
341}
342
343impl<T> EventTail<T>
344where
345 T: FramePayload,
346{
347 /// Publishes an event and wakes every stream parked on this tail.
348 ///
349 /// The returned id is the value a client quotes to resume after this event,
350 /// so a publisher that discards it has made the event unresumable: the next
351 /// reconnect is told about a gap the client never knew it had.
352 pub fn push(&self, payload: T) -> EventId {
353 let id = self.shared.ring().push(payload);
354 // The wakers are drained under the ring lock and woken outside it. The
355 // drain has to be inside so a stream cannot register between the push
356 // and the collection; the wake has to be outside so a waker that
357 // re-enters this tail cannot deadlock on the lock it is being woken
358 // while holding.
359 let wakers: Vec<Waker> = {
360 let _ring = self.shared.ring();
361 self.shared
362 .waiters()
363 .entries
364 .drain(..)
365 .map(|(_, waker)| waker)
366 .collect()
367 };
368 for waker in &wakers {
369 waker.wake_by_ref();
370 }
371 id
372 }
373
374 /// Decides what a client that sent `header` gets, atomically with the
375 /// watermark it was decided against.
376 ///
377 /// The pairing with the cursor is the point. `Resume` names the frames to
378 /// write; the stream needs to know where to *continue* from. Reading those
379 /// from two separate lock acquisitions would open the window the module docs
380 /// describe, so [`Planned`] carries both and there is no way to obtain one
381 /// without the other.
382 ///
383 /// `header` is the raw `Last-Event-ID` value, or `None` for a client that has
384 /// never seen this stream. A header that is present but unreadable is a
385 /// refusal, decided before the ring is consulted.
386 pub fn plan(&self, header: Option<&str>) -> Planned<T> {
387 let ring = self.shared.ring();
388 let resume = plan(&ring, header);
389 // The cursor is the last id the client is known to hold, which is the
390 // id the decision accounted for — except after a gap, where it is one
391 // _below_ the first id the ring can answer for.
392 //
393 // `resume_from` is the first surviving event, and the client has not
394 // seen it, so a cursor left there would ask for `resume_from` and be
395 // handed `resume_from + 1` onward: the one event that survived the drop
396 // and was not lost would be the one lost. `resume_from - 1` is the last
397 // id the client can be said to hold, and asking for it returns the
398 // survivors in full.
399 let cursor = match &resume {
400 Resume::Gap {
401 evicted_through, ..
402 } => *evicted_through,
403 other => other.last().unwrap_or(EventId::WATERMARK),
404 };
405 drop(ring);
406 Planned {
407 shared: Arc::clone(&self.shared),
408 resume,
409 cursor,
410 }
411 }
412}
413
414/// The response-building half of [`EventTail`], separated by its bound.
415///
416/// The bound is `T: Send + 'static` because the body it produces has to be sent
417/// between threads and outlive the call; [`push`](Self::push) and
418/// [`plan`](Self::plan) do not carry it, so a caller that only publishes or only
419/// inspects never has to satisfy it.
420impl<T> EventTail<T>
421where
422 T: FramePayload + Send + 'static,
423{
424 /// Plans and answers in one call, for a router with nothing to do in
425 /// between.
426 ///
427 /// # Errors
428 ///
429 /// [`StreamError::Response`] if the response headers cannot be assembled.
430 pub fn respond(
431 &self,
432 header: Option<&str>,
433 keep_alive: Duration,
434 ) -> Result<Response, StreamError> {
435 self.plan(header).into_response(keep_alive)
436 }
437}
438
439/// A decision, the watermark it was made against, and the tail to continue from.
440///
441/// Bundling these is what makes the replay-to-tail race unrepresentable rather
442/// than merely unlikely: there is no way to obtain a [`Resume`] and a starting
443/// cursor separately, so a caller cannot pair a decision with a cursor from a
444/// later read.
445#[derive(Debug)]
446pub struct Planned<T> {
447 shared: Arc<Shared<T>>,
448 resume: Resume,
449 cursor: EventId,
450}
451
452impl<T> Planned<T> {
453 /// The decision.
454 #[must_use]
455 pub const fn resume(&self) -> &Resume {
456 &self.resume
457 }
458
459 /// The cursor the stream will continue from.
460 ///
461 /// Every frame the decision did not include has an id strictly greater than
462 /// this, which is what makes a push between the decision and the first poll
463 /// undroppable.
464 #[must_use]
465 pub const fn cursor(&self) -> EventId {
466 self.cursor
467 }
468
469 /// The refusal, when this plan is one.
470 ///
471 /// For a router that wants to log or count the refusal separately from
472 /// answering it.
473 #[must_use]
474 pub const fn refusal(&self) -> Option<&super::Refusal> {
475 match &self.resume {
476 Resume::Refused(refusal) => Some(refusal),
477 Resume::Replay { .. } | Resume::Live { .. } | Resume::Gap { .. } => None,
478 }
479 }
480}
481
482impl<T> Planned<T> {
483 /// A plan that is nothing but a refusal, for a router that declined before
484 /// the ring was ever consulted.
485 ///
486 /// Separate from [`plan`](EventTail::plan) because a router refuses for
487 /// reasons the ring knows nothing about — no credentials, no such session,
488 /// wrong method, body too large — and every one of those has to come out as
489 /// the same shape as a cursor the ring refused, or a caller would have two
490 /// ways of producing a status and no reason to pick the right one.
491 ///
492 /// # Example
493 ///
494 /// ```
495 /// use core::num::NonZeroUsize;
496 /// use serde_json::Value;
497 /// use protocol_droid::adapters::primary::transport::http::{EventTail, Refusal, planned_refusal};
498 ///
499 /// let capacity = NonZeroUsize::new(4).ok_or("capacity")?;
500 /// let tail = EventTail::<Value>::new(capacity);
501 /// let planned = planned_refusal(&tail, Refusal::Unauthenticated);
502 /// assert!(planned.resume().pending().is_empty());
503 /// assert!(planned.cursor().is_watermark());
504 /// # Ok::<(), Box<dyn std::error::Error>>(())
505 /// ```
506 #[must_use]
507 pub fn refused(tail: &EventTail<T>, refusal: super::Refusal) -> Self {
508 Self {
509 shared: Arc::clone(&tail.shared),
510 resume: Resume::Refused(refusal),
511 // A refusal is not a stream, so it has no cursor. The watermark is
512 // the honest value: nothing has been written, so nothing has been
513 // consumed, and a caller that logged this number would be reporting
514 // "no events issued" rather than "no cursor".
515 cursor: EventId::WATERMARK,
516 }
517 }
518}
519
520/// A [`Planned`] that is a refusal, for a router that declined before the ring
521/// was consulted.
522///
523/// The free-function spelling of [`Planned::refused`], for a caller that has a
524/// `Refusal` and a tail and no other use for the plan.
525#[must_use]
526pub fn planned_refusal<T>(tail: &EventTail<T>, refusal: super::Refusal) -> Planned<T> {
527 Planned::refused(tail, refusal)
528}
529
530impl<T> Planned<T>
531where
532 T: FramePayload,
533{
534 /// The live body, or the refusal response when there is no stream to open.
535 ///
536 /// A refusal is never a stream. Returning it as an `Err` carrying a
537 /// [`Response`] — rather than as an `Option`, a sentinel frame, or an empty
538 /// body — keeps the two cases impossible to confuse: the success type is a
539 /// body and the failure type is a status code. The `Err` is a `Box` because
540 /// a `Response` is large next to the `EventStream` beside it, and an enum
541 /// sized to its largest arm would put that on every success path too.
542 pub fn try_stream(&self, keep_alive: Duration) -> Result<EventStream<T>, Box<Response>> {
543 if let Resume::Refused(refusal) = &self.resume {
544 return Err(Box::new(refusal.clone().into_response()));
545 }
546 Ok(EventStream {
547 shared: Arc::clone(&self.shared),
548 // The decision's frames are queued before the cursor is ever
549 // consulted, so a `Replay`'s last frame is not written twice and
550 // its first unseen frame is not skipped.
551 queued: self.resume.pending().iter().cloned().collect(),
552 cursor: self.cursor,
553 keep_alive,
554 deadline: None,
555 registration: None,
556 closed: false,
557 })
558 }
559
560 /// The response for a `POST`, which ends when the tail has been drained.
561 ///
562 /// The long-lived [`Self::into_response`] is the wrong shape for a
563 /// `POST`: it stays open so a client can receive messages the server
564 /// initiates after the reply, which is what a `GET` is for. A `POST` is
565 /// one request and one exchange, and a client that waits for its stream to
566 /// close waits forever — the request it sent is complete, and nothing else
567 /// is coming on *this* stream.
568 ///
569 /// So this closes once everything the handler published has been written,
570 /// which is exactly the set of frames a single request produced: the
571 /// notifications a tool emitted while it ran, and then the reply. Draining
572 /// rather than snapshotting is what lets a progress notification emitted
573 /// mid-call reach the client at all.
574 ///
575 /// # Errors
576 ///
577 /// Propagates whatever [`Self::try_stream`] does, so a plan that resolves
578 /// to a refusal is still answered with that refusal rather than an empty
579 /// stream.
580 pub fn into_draining_response(&self, keep_alive: Duration) -> Result<Response, StreamError> {
581 let stream = self
582 .try_stream(keep_alive)
583 .map_err(|_| StreamError::Response("the stream could not open".to_owned()))?;
584 stream_response(stream.finishing_when_drained())
585 }
586
587 /// The response for a `POST`, ending when `finished` is signalled.
588 ///
589 /// [`Self::into_draining_response`] ends when the tail runs dry, which is
590 /// right for a request that publishes and returns. It is wrong for one that
591 /// publishes and then *waits*: a tool that asks the peer to sample parks on
592 /// that answer, publishes nothing further while it waits, and a stream that
593 /// ends on an empty tail closes before the peer's reply — which arrives on
594 /// a different request entirely — can be read. The tool waits forever on a
595 /// request the client already stopped listening to.
596 ///
597 /// So the end is a fact about the *handler*, not about the buffer: the
598 /// caller signals it, and the stream lasts until it does.
599 pub fn into_finishable_response(
600 &self,
601 finished: Finishing,
602 keep_alive: Duration,
603 ) -> Result<Response, StreamError> {
604 let stream = self
605 .try_stream(keep_alive)
606 .map_err(|_| StreamError::Response("the stream could not open".to_owned()))?;
607 stream_response(stream.finishing_on(finished))
608 }
609}
610
611/// What ends a [`FinishingStream`].
612///
613/// An enum rather than a bare flag because the two ends answer different
614/// questions: "has the handler returned?" is about the task, and "is the tail
615/// empty?" is about the buffer. One flag can express only one of them, and
616/// picking the wrong one closes the stream at exactly the moment the other
617/// still matters.
618#[derive(Debug, Clone)]
619enum Until {
620 /// The tail has nothing left — the handler published and returned.
621 Drained,
622 /// The handler has returned, whether or not the tail is empty.
623 Signalled(Finishing),
624}
625
626/// The one-shot signal that a handler has returned.
627///
628/// A signal rather than a flag under a mutex so the stream *wakes* when it is
629/// set. A flag alone would leave a stream parked in `Pending` with no event to
630/// poll it, so the end would only be noticed on the next unrelated frame —
631/// which for a handler that returned without publishing is never.
632#[derive(Debug, Clone)]
633pub struct Finishing {
634 inner: Arc<FinishingInner>,
635}
636
637impl Finishing {
638 /// A signal nobody has fired yet.
639 #[must_use]
640 pub fn new() -> Self {
641 Self {
642 inner: Arc::new(FinishingInner::new()),
643 }
644 }
645
646 /// Whether the handler has returned.
647 #[must_use]
648 pub fn is_fired(&self) -> bool {
649 self.inner.fired.load(Ordering::SeqCst)
650 }
651
652 /// Signals that the handler has returned.
653 ///
654 /// Idempotent: a handler that returns twice — which a `catch_unwind` or a
655 /// dropped guard can arrange — must not panic, because "finished" is not
656 /// something with a legal second answer.
657 pub fn fire(&self) {
658 self.inner.fire();
659 }
660}
661
662impl Default for Finishing {
663 fn default() -> Self {
664 Self::new()
665 }
666}
667
668/// The shared half of a [`Finishing`].
669#[derive(Debug)]
670struct FinishingInner {
671 fired: std::sync::atomic::AtomicBool,
672 wakers: std::sync::Mutex<Vec<Waker>>,
673}
674
675impl FinishingInner {
676 const fn new() -> Self {
677 Self {
678 fired: std::sync::atomic::AtomicBool::new(false),
679 wakers: std::sync::Mutex::new(Vec::new()),
680 }
681 }
682
683 /// Records the fire, and hands back anyone waiting to hear about it.
684 fn fire(&self) {
685 self.fired.store(true, Ordering::SeqCst);
686 let wakers =
687 std::mem::take(&mut *self.wakers.lock().unwrap_or_else(PoisonError::into_inner));
688 for waker in wakers {
689 waker.wake();
690 }
691 }
692
693 /// Wakes `waker` when this fires, if it has not already.
694 fn subscribe(&self, waker: &Waker) {
695 if self.fired.load(Ordering::SeqCst) {
696 waker.wake_by_ref();
697 return;
698 }
699 self.wakers
700 .lock()
701 .unwrap_or_else(PoisonError::into_inner)
702 .push(waker.clone());
703 }
704}
705
706struct FinishingStream<T> {
707 /// The stream doing the work.
708 inner: EventStream<T>,
709 /// What ends it.
710 ///
711 /// An enum rather than a bare flag because the two ends answer different
712 /// questions: "has the handler finished?" is about the task, and "is the
713 /// tail empty?" is about the buffer. A single flag can express only one of
714 /// them, and picking the wrong one closes the stream at exactly the moment
715 /// the other still matters.
716 until: Until,
717}
718
719impl<T: FramePayload> Stream for FinishingStream<T> {
720 type Item = Result<Bytes, StreamError>;
721
722 fn poll_next(self: Pin<&mut Self>, context: &mut Context<'_>) -> Poll<Option<Self::Item>> {
723 let this = self.get_mut();
724 match Pin::new(&mut this.inner).poll_next(context) {
725 Poll::Ready(Some(frame)) => return Poll::Ready(Some(frame)),
726 Poll::Ready(None) => {
727 this.inner.finish();
728 return Poll::Ready(None);
729 }
730 Poll::Pending => {}
731 }
732
733 // Parked with nothing to write. Whether that is the end depends on
734 // what this stream is waiting for: a drained tail means the handler
735 // published its last frame and returned, but a handler still running
736 // can publish again at any moment — and closing then is exactly the
737 // deadlock above.
738 let ended = match &this.until {
739 Until::Drained => !this.inner.has_pending(),
740 Until::Signalled(finishing) => {
741 let ready = finishing.is_fired() && !this.inner.has_pending();
742 if !ready {
743 // Park until the handler returns, or until it publishes
744 // something the inner stream can write.
745 finishing.inner.subscribe(context.waker());
746 }
747 ready
748 }
749 };
750 if ended {
751 this.inner.finish();
752 return Poll::Ready(None);
753 }
754 Poll::Pending
755 }
756}
757
758/// The response-building half of [`Planned`], separated by its bound.
759///
760/// `T: Send + 'static` is required here and only here, because
761/// [`Body::from_stream`] requires the stream to be sent between threads and to
762/// outlive the call. Putting the bound on the whole impl would force it on
763/// [`resume`](Planned::resume) and [`cursor`](Planned::cursor), which a caller
764/// reads from a context that has no reason to have a `Send` payload in hand.
765impl<T> Planned<T>
766where
767 T: FramePayload + Send + 'static,
768{
769 /// The response to send, whatever the decision was.
770 ///
771 /// This is what a router calls. A `Refused` becomes a status and a JSON-RPC
772 /// body; every other arm becomes a `200 text/event-stream` whose body writes
773 /// the buffered frames and then stays open.
774 ///
775 /// # Errors
776 ///
777 /// [`StreamError::Response`] if the response headers cannot be assembled.
778 pub fn into_response(self, keep_alive: Duration) -> Result<Response, StreamError> {
779 if let Resume::Refused(refusal) = &self.resume {
780 // `Refusal`'s own `IntoResponse` is the single place its status and
781 // body are defined. Re-deriving them here would be a second opinion
782 // about a wire contract a client branches on. The clone is of a
783 // three-field enum, once per refused request.
784 return Ok(refusal.clone().into_response());
785 }
786 let stream = EventStream {
787 shared: Arc::clone(&self.shared),
788 queued: self.resume.pending().iter().cloned().collect(),
789 cursor: self.cursor,
790 keep_alive,
791 deadline: None,
792 registration: None,
793 closed: false,
794 };
795 stream_response(stream)
796 }
797}
798
799/// A live `text/event-stream` body.
800///
801/// Implements [`Stream`] directly rather than going through axum's `Sse`
802/// helper because this stream has to emit a frame the helper has no spelling
803/// for: a frame carrying no `id:` is exactly the gap notice, and building it
804/// through a type whose every event implies a cursor is how a gap notice comes
805/// to carry one.
806///
807/// The item type is a `Result` because [`Body::from_stream`] requires it, not
808/// because a frame can fail — by the time a frame is queued it is already
809/// encoded.
810#[derive(Debug)]
811pub struct EventStream<T> {
812 shared: Arc<Shared<T>>,
813 queued: VecDeque<SseEvent>,
814 cursor: EventId,
815 keep_alive: Duration,
816 deadline: Option<Pin<Box<tokio::time::Sleep>>>,
817 registration: Option<u64>,
818 closed: bool,
819}
820
821impl<T> EventStream<T> {
822 /// The cursor this stream has written up to.
823 #[must_use]
824 pub const fn cursor(&self) -> EventId {
825 self.cursor
826 }
827
828 /// Whether a waker is currently registered with the tail.
829 ///
830 /// The subscription is held only while the stream is parked, which is what
831 /// keeps a client that disconnects between frames from leaving anything
832 /// behind.
833 #[must_use]
834 pub const fn is_subscribed(&self) -> bool {
835 self.registration.is_some()
836 }
837
838 /// Whether a frame is waiting to be written.
839 ///
840 /// Distinct from "the tail is empty": a frame can be buffered while the
841 /// tail is already spent, which is the state a stream sits in between
842 /// writing one frame and being polled for the next.
843 #[must_use]
844 pub fn has_pending(&self) -> bool {
845 !self.queued.is_empty()
846 }
847
848 /// Ends this stream after whatever it has already written.
849 ///
850 /// Only meaningful for a stream that will not be written to again — a
851 /// `POST`'s. Calling it on a long-lived `GET` stream truncates the
852 /// connection the client is holding open, which is why
853 /// [`Self::finishing_when_drained`] is the only caller and it calls only
854 /// when the tail is spent.
855 fn finish(&mut self) {
856 self.unregister();
857 self.closed = true;
858 }
859
860 /// This stream, ending once the tail is drained.
861 ///
862 /// See [`FinishingStream`] for why a `POST` needs this and a `GET` does not.
863 const fn finishing_when_drained(self) -> FinishingStream<T> {
864 FinishingStream {
865 inner: self,
866 until: Until::Drained,
867 }
868 }
869
870 /// This stream, ending when `finished` is signalled.
871 const fn finishing_on(self, finished: Finishing) -> FinishingStream<T> {
872 FinishingStream {
873 inner: self,
874 until: Until::Signalled(finished),
875 }
876 }
877
878 /// Removes this stream's registration, if it has one.
879 ///
880 /// Unbounded over `T` because [`Drop`] is: a `Drop` impl cannot add a bound
881 /// the struct does not carry, so a method that only the destructor calls has
882 /// to be available for every `T` the stream is instantiated with.
883 fn unregister(&mut self) {
884 let Some(id) = self.registration.take() else {
885 return;
886 };
887 self.shared
888 .waiters()
889 .entries
890 .retain(|(entry, _)| *entry != id);
891 }
892}
893
894impl<T> EventStream<T>
895where
896 T: FramePayload,
897{
898 /// The next decision, and the waker registered under the ring lock when it
899 /// is "nothing yet".
900 ///
901 /// The read and the registration share one critical section on purpose. A
902 /// stream that found nothing and registered its waker did both while the
903 /// ring lock was held, so a concurrent [`push`](EventTail::push) either
904 /// finished before the read — and the frame is in the frames we return — or
905 /// blocks on the lock and then finds the waker already in the list. There is
906 /// no interleaving in which a frame is pushed and nobody is listening.
907 fn take(&mut self, context: &Context<'_>) -> Batch {
908 let ring = self.shared.ring();
909 let decision = decide(&ring.replay_from(Some(self.cursor)));
910 let batch = match decision {
911 Resume::Replay { frames, last } => Batch {
912 frames,
913 cursor: last,
914 parked: false,
915 closed: false,
916 },
917 // Nothing new and nothing lost. The cursor is already the watermark
918 // — `replay_from` only answers `UpToDate` when the requested id
919 // *is* the last — so it does not move.
920 Resume::Live { .. } => Batch::parked(),
921 // The notice goes out first, and the cursor lands on `resume_from`
922 // — the first id the ring can answer for. Not the watermark, which
923 // would drop the frames the ring still holds in silence, and not
924 // `evicted_through`, which the ring's gap test is inclusive of and
925 // would therefore gap again on the very next poll, leaving the
926 // client a stream of notices and no events.
927 Resume::Gap {
928 notice,
929 resume_from,
930 ..
931 } => Batch {
932 frames: vec![notice],
933 cursor: resume_from,
934 parked: false,
935 closed: false,
936 },
937 // Both refusals mean there is no stream to continue: the ring was
938 // cleared under this client, or a payload had no JSON form. The
939 // client holds a healthy connection, so the honest signal is a
940 // stream end, and it will reconnect and be told why.
941 Resume::Refused(_) => Batch {
942 frames: Vec::new(),
943 cursor: self.cursor,
944 parked: false,
945 closed: true,
946 },
947 };
948 if batch.parked {
949 // `&self` rather than `&mut self` precisely so this can happen
950 // while the ring guard is alive. That is the anti-race mechanism:
951 // the read that found nothing and the registration that will hear
952 // about the next push are one critical section, so there is no
953 // instant at which a frame could be pushed with nothing listening.
954 let id = self.registration(context);
955 self.registration = Some(id);
956 }
957 drop(ring);
958 batch
959 }
960
961 /// Registers this stream's waker, or refreshes the one it already has, and
962 /// returns the id it is registered under.
963 ///
964 /// One entry per stream, updated in place. A stream polls in a loop and a
965 /// re-registration on every `Pending` would grow the list without bound,
966 /// and — worse — would leave a `Drop` that removes only the newest id
967 /// behind every earlier one.
968 fn registration(&self, context: &Context<'_>) -> u64 {
969 let waker = context.waker();
970 let mut waiters = self.shared.waiters();
971 if let Some(id) = self.registration
972 && let Some((_, registered)) =
973 waiters.entries.iter_mut().find(|(entry, _)| *entry == id)
974 {
975 // The same waker: nothing to do, and the existing id stands. A
976 // different one means the stream was moved between tasks, and
977 // the entry has to follow it or the old task is woken for a
978 // stream it no longer polls. `clone_from` reuses the existing
979 // allocation where the two hold the same kind of waker, which a
980 // plain assign-then-clone cannot.
981 if !registered.will_wake(waker) {
982 registered.clone_from(waker);
983 }
984 return id;
985 }
986 let fresh = self.shared.next_registration();
987 waiters.entries.push((fresh, waker.clone()));
988 fresh
989 }
990
991 /// Arms the keep-alive deadline, if none is armed.
992 ///
993 /// Armed here rather than in the constructor because `tokio::time::sleep`
994 /// needs a runtime context and a stream is often constructed outside one.
995 /// A stream with frames to write never reaches here — a keep-alive on a busy
996 /// stream is noise — so this is only reached by a stream that has nothing to
997 /// say, which is exactly the stream that needs to be told it is still there.
998 ///
999 /// The `try_current` check is not defensive noise: `tokio::time::sleep`
1000 /// **panics** with no current runtime, and this function is reachable from a
1001 /// plain `poll_next` on any executor. A `Stream` that only works on tokio
1002 /// would be a latent panic in every embedder that drives it elsewhere, and
1003 /// the failure would surface as a panic inside a poll rather than as a
1004 /// typed error a caller could handle. Without a runtime there is no timer to
1005 /// arm, so the stream stays silent — which is the correct degradation,
1006 /// because the alternative is a heartbeat that cannot be scheduled.
1007 fn arm_keep_alive(&mut self) {
1008 if self.deadline.is_some() {
1009 return;
1010 }
1011 if tokio::runtime::Handle::try_current().is_err() {
1012 return;
1013 }
1014 self.deadline = Some(Box::pin(tokio::time::sleep(self.keep_alive)));
1015 }
1016}
1017
1018impl<T> Drop for EventStream<T> {
1019 /// Releases the subscription.
1020 ///
1021 /// In `Drop` and not in a method because a client disconnecting mid-stream
1022 /// is the *normal* way a stream ends, and after the response body is
1023 /// dropped there is no code path left for anyone to call a cleanup on. A
1024 /// registration removed anywhere other than here would be removed on a path
1025 /// a dropped client does not take.
1026 fn drop(&mut self) {
1027 self.unregister();
1028 }
1029}
1030
1031/// What one pass over the ring decided.
1032struct Batch {
1033 /// The frames to write, in order.
1034 frames: Vec<SseEvent>,
1035 /// Where the cursor goes once these frames are queued.
1036 cursor: EventId,
1037 /// Whether there is nothing to write and the stream should park.
1038 parked: bool,
1039 /// Whether the stream is finished.
1040 closed: bool,
1041}
1042
1043impl Batch {
1044 /// Nothing to write, and the stream should wait.
1045 ///
1046 /// The cursor is the watermark, which is also where the stream's own cursor
1047 /// already is: `replay_from` answers `UpToDate` only when the requested id
1048 /// equals the newest. The field is set rather than left uninitialised only
1049 /// because the struct has no `Default`, and a `Default` here would be a
1050 /// second answer to "what is an empty batch" that could disagree with this
1051 /// one.
1052 const fn parked() -> Self {
1053 Self {
1054 frames: Vec::new(),
1055 cursor: EventId::WATERMARK,
1056 parked: true,
1057 closed: false,
1058 }
1059 }
1060}
1061
1062impl<T> Stream for EventStream<T>
1063where
1064 T: FramePayload,
1065{
1066 type Item = Result<Bytes, StreamError>;
1067
1068 fn poll_next(self: Pin<&mut Self>, context: &mut Context<'_>) -> Poll<Option<Self::Item>> {
1069 let this = self.get_mut();
1070 loop {
1071 if this.closed {
1072 return Poll::Ready(None);
1073 }
1074 if let Some(frame) = this.queued.pop_front() {
1075 // A waker is only wanted while there is nothing to write, and a
1076 // stream that is producing frames is polled again immediately,
1077 // so the registration is released rather than left to go stale.
1078 this.unregister();
1079 return Poll::Ready(Some(Ok(Bytes::from(frame.encode()))));
1080 }
1081
1082 let batch = this.take(context);
1083 if batch.closed {
1084 this.unregister();
1085 this.closed = true;
1086 return Poll::Ready(None);
1087 }
1088 if !batch.frames.is_empty() {
1089 this.cursor = batch.cursor;
1090 this.queued.extend(batch.frames);
1091 continue;
1092 }
1093
1094 this.arm_keep_alive();
1095 if let Some(deadline) = this.deadline.as_mut()
1096 && deadline.as_mut().poll(context).is_ready()
1097 {
1098 this.deadline = None;
1099 this.unregister();
1100 return Poll::Ready(Some(Ok(Bytes::from_static(KEEP_ALIVE_FRAME.as_bytes()))));
1101 }
1102 return Poll::Pending;
1103 }
1104 }
1105}
1106
1107/// Assembles the `200 text/event-stream` response around a body.
1108///
1109/// A free function so [`Planned::into_response`] and a test can both build the
1110/// header set, and so there is exactly one place that knows what a resumable
1111/// response looks like on the wire.
1112///
1113/// # Errors
1114///
1115/// [`StreamError::Response`] if the response headers cannot be assembled.
1116pub(super) fn stream_response<S>(stream: S) -> Result<Response, StreamError>
1117where
1118 S: Stream<Item = Result<Bytes, StreamError>> + Send + 'static,
1119{
1120 // Three headers, and each is load-bearing in a way that is invisible from
1121 // the frame encoder:
1122 //
1123 // - `Content-Type: text/event-stream` is what makes a client parse this as
1124 // a stream at all. Without it the same bytes are an opaque body.
1125 // - `Cache-Control: no-cache` is what stops an intermediary from serving a
1126 // stored response to the *next* client, which for a stream means that
1127 // client receives the previous one's events.
1128 // - `X-Accel-Buffering: no` is nginx's opt-out. Without it nginx buffers
1129 // the response and delivers it in lumps, which presents as a live server
1130 // that has gone silent.
1131 Response::builder()
1132 .status(StatusCode::OK)
1133 .header(header::CONTENT_TYPE, EVENT_STREAM)
1134 .header(header::CACHE_CONTROL, "no-cache")
1135 .header(ACCEL_BUFFERING, "no")
1136 .body(Body::from_stream(stream))
1137 .map_err(StreamError::from)
1138}
1139
1140/// How many streams are currently parked on a tail.
1141///
1142/// For a router's shutdown path, and for asserting that a dropped body released
1143/// its subscription. A non-zero count after the last response has been dropped
1144/// is a leak.
1145///
1146/// # Example
1147///
1148/// ```
1149/// use core::num::NonZeroUsize;
1150/// use serde_json::Value;
1151/// use protocol_droid::adapters::primary::transport::http::{EventTail, parked_streams};
1152///
1153/// let capacity = NonZeroUsize::new(4).ok_or("capacity")?;
1154/// let tail = EventTail::<Value>::new(capacity);
1155/// assert_eq!(parked_streams(&tail), 0);
1156/// # Ok::<(), Box<dyn std::error::Error>>(())
1157/// ```
1158#[must_use]
1159pub fn parked_streams<T>(tail: &EventTail<T>) -> usize {
1160 tail.shared.waiters().entries.len()
1161}
1162
1163#[cfg(test)]
1164mod tests {
1165 use core::num::NonZeroUsize;
1166 use core::pin::Pin;
1167 use core::task::{Context, Poll, Waker};
1168 use core::time::Duration;
1169 use std::fmt;
1170 use std::sync::Arc;
1171 use std::sync::atomic::{AtomicUsize, Ordering};
1172 use std::task::Wake;
1173
1174 use axum::body::Bytes;
1175 use axum::http::StatusCode;
1176 use futures_core::Stream;
1177 use serde_json::{Value, json};
1178
1179 use super::{
1180 EventStream, EventTail, FramePayload, KEEP_ALIVE_FRAME, StreamError, parked_streams,
1181 };
1182 use crate::adapters::primary::transport::http::{
1183 Refusal, Resume, SseError, SseEvent, planned_refusal,
1184 };
1185 use crate::domain::EventId;
1186
1187 /// Why a test failed, as one error type the `?` can produce.
1188 ///
1189 /// The fallible calls in this module answer unrelated error types — an
1190 /// `Option` from `NonZeroUsize`, an `SseError` from a decode, a
1191 /// [`StreamError`] from a poll — and a test that collected them in a `Box`
1192 /// would have to name each. One enum with a `From` each is the same thing
1193 /// with the conversions written once.
1194 #[derive(Debug)]
1195 enum Failure {
1196 /// A capacity of zero, which `NonZeroUsize` cannot represent.
1197 ZeroCapacity,
1198 /// A plan produced a stream when the test needed a refusal response.
1199 UnexpectedStream,
1200 /// A decode that failed.
1201 Sse(SseError),
1202 /// A frame that was not the shape the test claims it must be.
1203 Wrong(String),
1204 }
1205
1206 impl fmt::Display for Failure {
1207 fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
1208 match self {
1209 Self::ZeroCapacity => {
1210 write!(formatter, "a ring capacity of zero is not a capacity")
1211 }
1212 Self::UnexpectedStream => write!(formatter, "expected a refusal, got a stream"),
1213 Self::Sse(error) => write!(formatter, "sse: {error}"),
1214 Self::Wrong(detail) => write!(formatter, "{detail}"),
1215 }
1216 }
1217 }
1218
1219 impl std::error::Error for Failure {}
1220
1221 impl From<SseError> for Failure {
1222 fn from(error: SseError) -> Self {
1223 Self::Sse(error)
1224 }
1225 }
1226
1227 impl From<StreamError> for Failure {
1228 fn from(error: StreamError) -> Self {
1229 Self::Wrong(error.to_string())
1230 }
1231 }
1232
1233 /// The one error type these tests return.
1234 type TestError = Failure;
1235
1236 /// A waker that counts how many times it was woken.
1237 ///
1238 /// A count rather than a flag because the properties under test care
1239 /// whether the *right* wake happened, and "it woke at least once" is the
1240 /// only claim a flag can make.
1241 #[derive(Default)]
1242 struct Counting {
1243 wakes: AtomicUsize,
1244 }
1245
1246 impl Counting {
1247 fn wakes(&self) -> usize {
1248 self.wakes.load(Ordering::SeqCst)
1249 }
1250
1251 fn waker(self: &Arc<Self>) -> Waker {
1252 Waker::from(Arc::clone(self))
1253 }
1254 }
1255
1256 impl Wake for Counting {
1257 fn wake(self: Arc<Self>) {
1258 self.wakes.fetch_add(1, Ordering::SeqCst);
1259 }
1260
1261 fn wake_by_ref(self: &Arc<Self>) {
1262 self.wakes.fetch_add(1, Ordering::SeqCst);
1263 }
1264 }
1265
1266 /// A tail retaining `capacity` events of JSON payload.
1267 fn tail(capacity: usize) -> Result<EventTail<Value>, TestError> {
1268 let capacity = NonZeroUsize::new(capacity).ok_or(Failure::ZeroCapacity)?;
1269 Ok(EventTail::new(capacity))
1270 }
1271
1272 /// Polls a stream once with a caller-supplied waker.
1273 ///
1274 /// Returns the poll rather than a decoded frame, because the properties
1275 /// under test are about *whether* a poll produced bytes and about the exact
1276 /// bytes it produced — decoding inside the harness would hide both.
1277 fn poll_once<T: FramePayload>(
1278 stream: &mut EventStream<T>,
1279 waker: &Waker,
1280 ) -> Poll<Option<Result<Bytes, StreamError>>> {
1281 let mut context = Context::from_waker(waker);
1282 core::pin::Pin::new(stream).poll_next(&mut context)
1283 }
1284
1285 /// The bytes of the frame a poll produced, or a description of why there
1286 /// were none.
1287 fn bytes_of(poll: Poll<Option<Result<Bytes, StreamError>>>) -> Result<String, TestError> {
1288 match poll {
1289 Poll::Ready(Some(Ok(bytes))) => Ok(String::from_utf8_lossy(&bytes).into_owned()),
1290 Poll::Ready(Some(Err(error))) => Err(Failure::Wrong(error.to_string())),
1291 Poll::Ready(None) => Err(Failure::Wrong("the stream ended".to_owned())),
1292 Poll::Pending => Err(Failure::Wrong("the stream parked".to_owned())),
1293 }
1294 }
1295
1296 /// The decoded event a frame carries.
1297 fn event_of(text: &str) -> Result<SseEvent, TestError> {
1298 Ok(SseEvent::decode(text)?)
1299 }
1300
1301 /// The `id:` a frame carries, or the watermark for one that carries none.
1302 ///
1303 /// `WATERMARK` rather than an `Option` because a frame with no id and a
1304 /// frame at id 0 mean the same thing on a wire, and the tests that care
1305 /// about the *absence* read `SseEvent::id` directly.
1306 fn id_of(event: &SseEvent) -> EventId {
1307 event.id().unwrap_or(EventId::WATERMARK)
1308 }
1309
1310 /// Asserts a frame's decoded payload is the JSON value a test expects.
1311 fn assert_payload(event: &SseEvent, expected: &Value) -> Result<(), TestError> {
1312 let actual: Value = serde_json::from_str(event.data())
1313 .map_err(|error| Failure::Wrong(error.to_string()))?;
1314 if &actual == expected {
1315 Ok(())
1316 } else {
1317 Err(Failure::Wrong(format!(
1318 "expected payload {expected}, got {actual}"
1319 )))
1320 }
1321 }
1322
1323 /// A long keep-alive, so a test that does not want one never sees one.
1324 fn no_keep_alive() -> Duration {
1325 Duration::from_mins(15)
1326 }
1327
1328 // ---------------------------------------------------------------------
1329 // A `Replay` is written whole, in order, with its cursors.
1330 // ---------------------------------------------------------------------
1331
1332 /// A resuming client gets every frame after its cursor, in order, with
1333 /// every `id:` intact.
1334 ///
1335 /// The mutation this kills is writing the window in reverse, or dropping
1336 /// the `id:` while reordering — both produce a stream that looks complete
1337 /// and cannot be resumed, because the client's next reconnect quotes
1338 /// whatever the last frame it received said.
1339 #[test]
1340 fn the_buffered_frames_come_out_in_order() -> Result<(), TestError> {
1341 let tail = tail(8)?;
1342 for index in 1..=3 {
1343 tail.push(json!({ "index": index }));
1344 }
1345
1346 let planned = tail.plan(Some("0"));
1347 let mut stream = planned
1348 .try_stream(no_keep_alive())
1349 .map_err(|_| Failure::UnexpectedStream)?;
1350 let counting = Arc::new(Counting::default());
1351 let waker = counting.waker();
1352
1353 for index in 1..=3u64 {
1354 let frame = bytes_of(poll_once(&mut stream, &waker))?;
1355 let event = event_of(&frame)?;
1356 if id_of(&event) != EventId::new(index) {
1357 return Err(Failure::Wrong(format!(
1358 "expected frame {index}, got {}",
1359 id_of(&event).get()
1360 )));
1361 }
1362 assert_payload(&event, &json!({ "index": index }))?;
1363 }
1364 Ok(())
1365 }
1366
1367 // ---------------------------------------------------------------------
1368 // A gap notice comes first, carries no id, and does not skip the rest.
1369 // ---------------------------------------------------------------------
1370
1371 /// A client behind the eviction point is told what it missed *before* any
1372 /// live frame, and the notice moves no cursor.
1373 ///
1374 /// The order is the load-bearing half. A writer that drained the ring
1375 /// before writing the notice would send a contiguous-looking stream in
1376 /// which the gap is invisible — the exact silent loss the whole module
1377 /// exists to prevent.
1378 ///
1379 /// The absence of an `id:` is the other half: a notice *with* a cursor
1380 /// would advance the client's resume point past the events it never
1381 /// received, making them unrecoverable rather than merely missed.
1382 #[test]
1383 fn the_gap_notice_precedes_the_live_frames_and_carries_no_id() -> Result<(), TestError> {
1384 let tail = tail(2)?;
1385 // Push three into a ring of two: the first is evicted, which is what
1386 // puts a real gap between what a client at 0 asked for and what
1387 // survives.
1388 for index in 1..=3 {
1389 tail.push(json!({ "index": index }));
1390 }
1391
1392 let planned = tail.plan(Some("0"));
1393 let mut stream = planned
1394 .try_stream(no_keep_alive())
1395 .map_err(|_| Failure::UnexpectedStream)?;
1396 let counting = Arc::new(Counting::default());
1397 let waker = counting.waker();
1398
1399 let notice = event_of(&bytes_of(poll_once(&mut stream, &waker))?)?;
1400 // The notice is a control frame, not an event the client can resume
1401 // from. `id()` returning `None` is the claim; the watermark is what a
1402 // client would resume from if it wrongly took the notice's absence as
1403 // a cursor.
1404 if notice.id().is_some() {
1405 return Err(Failure::Wrong(
1406 "a gap notice must carry no id: it would move the client's \
1407 resume cursor past events it never received"
1408 .to_owned(),
1409 ));
1410 }
1411 let payload: Value = serde_json::from_str(notice.data())
1412 .map_err(|error| Failure::Wrong(error.to_string()))?;
1413 if payload.get("type").and_then(Value::as_str) != Some("resume-gap") {
1414 return Err(Failure::Wrong(format!(
1415 "expected a resume-gap notice, got {payload}"
1416 )));
1417 }
1418
1419 // The frames the ring still holds come *after* the notice, and they are
1420 // not skipped: the cursor lands on the eviction point, so the next poll
1421 // asks for exactly what survived.
1422 for index in 2..=3u64 {
1423 let event = event_of(&bytes_of(poll_once(&mut stream, &waker))?)?;
1424 if id_of(&event) != EventId::new(index) {
1425 return Err(Failure::Wrong(format!(
1426 "expected frame {index} after the notice, got {}",
1427 id_of(&event).get()
1428 )));
1429 }
1430 }
1431 Ok(())
1432 }
1433
1434 /// A client that falls behind *while connected* is told, not skipped.
1435 ///
1436 /// The mid-stream analogue of the test above, and the one that proves the
1437 /// stream reuses `decide` rather than a second answer written for the live
1438 /// path. A writer that only consulted the ring for *new* events would skip
1439 /// the evicted ones in silence, and the client would never know.
1440 #[test]
1441 fn a_client_that_falls_behind_mid_stream_is_told_rather_than_skipped() -> Result<(), TestError>
1442 {
1443 let tail = tail(2)?;
1444 tail.push(json!({ "index": 1 }));
1445 tail.push(json!({ "index": 2 }));
1446
1447 let planned = tail.plan(Some("2"));
1448 let mut stream = planned
1449 .try_stream(no_keep_alive())
1450 .map_err(|_| Failure::UnexpectedStream)?;
1451 let counting = Arc::new(Counting::default());
1452 let waker = counting.waker();
1453
1454 // The client is current and the stream parks.
1455 if !matches!(poll_once(&mut stream, &waker), Poll::Pending) {
1456 return Err(Failure::Wrong(
1457 "a current client should have parked, not produced bytes".to_owned(),
1458 ));
1459 }
1460
1461 // It now falls behind: three more pushes evict past the cursor it is
1462 // holding.
1463 for index in 3..=5 {
1464 tail.push(json!({ "index": index }));
1465 }
1466
1467 let notice = event_of(&bytes_of(poll_once(&mut stream, &waker))?)?;
1468 if notice.id().is_some() {
1469 return Err(Failure::Wrong(
1470 "a mid-stream gap notice must carry no id either".to_owned(),
1471 ));
1472 }
1473 let payload: Value = serde_json::from_str(notice.data())
1474 .map_err(|error| Failure::Wrong(error.to_string()))?;
1475 if payload.get("type").and_then(Value::as_str) != Some("resume-gap") {
1476 return Err(Failure::Wrong(format!(
1477 "a client that fell behind mid-stream was not told: got {payload}"
1478 )));
1479 }
1480 Ok(())
1481 }
1482
1483 // ---------------------------------------------------------------------
1484 // A `Live` is silence, not a close and not a comment.
1485 // ---------------------------------------------------------------------
1486
1487 /// A client that is already current gets no bytes at all.
1488 ///
1489 /// The three things this rules out, and each is a real mistake: a close
1490 /// (the client reconnects in a loop, forever), an empty event (the client
1491 /// dispatches a message the specification never describes), and a keep-alive
1492 /// comment (which is legitimate but is a *timeout*, and firing it
1493 /// immediately would mean the interval is not a heartbeat at all).
1494 ///
1495 /// The claim is about the poll that *follows* the plan, not about
1496 /// `Resume::pending` — a writer that queued a keep-alive would pass a test
1497 /// asserting only the latter.
1498 #[test]
1499 fn a_live_stream_is_silent_until_something_happens() -> Result<(), TestError> {
1500 let tail = tail(4)?;
1501 tail.push(json!({ "index": 1 }));
1502 tail.push(json!({ "index": 2 }));
1503
1504 let planned = tail.plan(Some("2"));
1505 if !planned.resume().pending().is_empty() {
1506 return Err(Failure::Wrong(
1507 "a current client must have nothing queued to write".to_owned(),
1508 ));
1509 }
1510 let mut stream = planned
1511 .try_stream(no_keep_alive())
1512 .map_err(|_| Failure::UnexpectedStream)?;
1513 let counting = Arc::new(Counting::default());
1514 let waker = counting.waker();
1515
1516 for _ in 0..3 {
1517 match poll_once(&mut stream, &waker) {
1518 Poll::Pending => {}
1519 Poll::Ready(_) => {
1520 return Err(Failure::Wrong(
1521 "a current client received bytes before anything happened".to_owned(),
1522 ));
1523 }
1524 }
1525 }
1526 Ok(())
1527 }
1528
1529 // ---------------------------------------------------------------------
1530 // A refusal is a status and a body, never a stream.
1531 // ---------------------------------------------------------------------
1532
1533 /// A malformed cursor is answered with a status and a JSON-RPC error
1534 /// document, and never as a stream.
1535 ///
1536 /// The mutation this kills is the one that makes a broken client look
1537 /// healthy: a `200 text/event-stream` whose first frame carries the error.
1538 /// Such a client is told it is connected, and then has to parse an error
1539 /// out of a data frame to discover it is not — while a well-behaved
1540 /// reconnecting client treats the same response as a live stream and hangs
1541 /// on it.
1542 #[tokio::test]
1543 async fn a_refusal_is_a_status_and_a_json_rpc_body_not_a_stream() -> Result<(), TestError> {
1544 let tail = tail(4)?;
1545 let response = tail
1546 .respond(Some("not-a-cursor"), no_keep_alive())
1547 .map_err(|error| Failure::Wrong(error.to_string()))?;
1548
1549 if response.status() != StatusCode::BAD_REQUEST {
1550 return Err(Failure::Wrong(format!(
1551 "a malformed cursor is a 400, got {}",
1552 response.status()
1553 )));
1554 }
1555 let content_type = response
1556 .headers()
1557 .get(axum::http::header::CONTENT_TYPE)
1558 .and_then(|value| value.to_str().ok())
1559 .unwrap_or_default();
1560 if content_type.starts_with(super::EVENT_STREAM) {
1561 return Err(Failure::Wrong(format!(
1562 "a refusal must not be served as {content_type}"
1563 )));
1564 }
1565
1566 let body = axum::body::to_bytes(response.into_body(), 4096)
1567 .await
1568 .map_err(|error| Failure::Wrong(error.to_string()))?;
1569 let document: Value =
1570 serde_json::from_slice(&body).map_err(|error| Failure::Wrong(error.to_string()))?;
1571 if document.get("jsonrpc").and_then(Value::as_str) != Some("2.0") {
1572 return Err(Failure::Wrong(format!(
1573 "a refusal body must be a JSON-RPC document, got {document}"
1574 )));
1575 }
1576 let error = document
1577 .get("error")
1578 .ok_or_else(|| Failure::Wrong("a refusal body must carry an error".to_owned()))?;
1579 if error.get("code").and_then(Value::as_i64).is_none() {
1580 return Err(Failure::Wrong(
1581 "a refusal body must carry a numeric error code".to_owned(),
1582 ));
1583 }
1584 if error.get("message").and_then(Value::as_str).is_none() {
1585 return Err(Failure::Wrong(
1586 "a refusal body must carry an error message".to_owned(),
1587 ));
1588 }
1589 Ok(())
1590 }
1591
1592 /// Every refusal arm becomes a non-`200`, whatever produced it.
1593 ///
1594 /// The single-arm test above proves the mechanism for a refusal the *ring*
1595 /// raised. This proves it for the ones it never sees: a router declines on
1596 /// credentials or a missing session before the ring is consulted, and a
1597 /// new [`Refusal`] variant that reached a client as a `200 text/event-stream`
1598 /// would be the exact failure this module exists to make unrepresentable.
1599 #[test]
1600 fn every_refusal_becomes_a_non_200() -> Result<(), TestError> {
1601 let refusals = [
1602 Refusal::MalformedCursor,
1603 Refusal::MalformedSessionId,
1604 Refusal::MissingSession,
1605 Refusal::Unauthenticated,
1606 Refusal::Forbidden,
1607 Refusal::UnknownCursor {
1608 requested: EventId::new(4),
1609 last: EventId::new(9),
1610 },
1611 ];
1612 for refusal in refusals {
1613 let tail = tail(4)?;
1614 let planned = planned_refusal(&tail, refusal.clone());
1615 if !planned.resume().pending().is_empty() {
1616 return Err(Failure::Wrong(format!(
1617 "{refusal} queued frames for a stream it will never open"
1618 )));
1619 }
1620 let response = planned
1621 .into_response(no_keep_alive())
1622 .map_err(|error| Failure::Wrong(error.to_string()))?;
1623 if response.status().is_success() {
1624 return Err(Failure::Wrong(format!("{refusal} was served as a success")));
1625 }
1626 let content_type = response
1627 .headers()
1628 .get(axum::http::header::CONTENT_TYPE)
1629 .and_then(|value| value.to_str().ok())
1630 .unwrap_or_default();
1631 if content_type.starts_with(super::EVENT_STREAM) {
1632 return Err(Failure::Wrong(format!(
1633 "{refusal} was served as {content_type}"
1634 )));
1635 }
1636 }
1637 Ok(())
1638 }
1639
1640 // ---------------------------------------------------------------------
1641 // The race: a push after the replay is not dropped.
1642 // ---------------------------------------------------------------------
1643
1644 /// A frame pushed between the plan and the first poll is delivered.
1645 ///
1646 /// This is the whole reason the live tail is a cursor. The window is
1647 /// genuinely open in the test — the ring is read, the decision is made, the
1648 /// response is planned, and only *then* is the frame published — which is
1649 /// the same ordering a real server has between reading the ring and the
1650 /// body being polled by hyper.
1651 ///
1652 /// The mutation this kills is the obvious one: a writer that drains the
1653 /// plan's frames and then only reacts to *future* pushes. The frame
1654 /// published in the window belongs to neither set, and the client resumes
1655 /// at 4 having never seen 4.
1656 #[test]
1657 fn a_frame_pushed_after_the_replay_is_not_dropped() -> Result<(), TestError> {
1658 let tail = tail(8)?;
1659 for index in 1..=3 {
1660 tail.push(json!({ "index": index }));
1661 }
1662
1663 // Decide. The cursor is now 3.
1664 let planned = tail.plan(Some("0"));
1665 if planned.cursor() != EventId::new(3) {
1666 return Err(Failure::Wrong(format!(
1667 "the cursor should be the watermark 3, got {}",
1668 planned.cursor().get()
1669 )));
1670 }
1671
1672 // The race: published in the window between the decision and the body
1673 // being read.
1674 tail.push(json!({ "index": 4 }));
1675
1676 let mut stream = planned
1677 .try_stream(no_keep_alive())
1678 .map_err(|_| Failure::UnexpectedStream)?;
1679 let counting = Arc::new(Counting::default());
1680 let waker = counting.waker();
1681
1682 // The replayed window, in order.
1683 for index in 1..=3u64 {
1684 let event = event_of(&bytes_of(poll_once(&mut stream, &waker))?)?;
1685 if id_of(&event) != EventId::new(index) {
1686 return Err(Failure::Wrong(format!(
1687 "expected replayed frame {index}, got {}",
1688 id_of(&event).get()
1689 )));
1690 }
1691 }
1692
1693 // And the frame from the window. If it is missing, the poll either
1694 // parked or produced something else, and both are reported as what
1695 // they are rather than as a generic assertion failure.
1696 let raced = event_of(&bytes_of(poll_once(&mut stream, &waker))?)?;
1697 if id_of(&raced) != EventId::new(4) {
1698 return Err(Failure::Wrong(format!(
1699 "the frame published in the replay-to-tail window was lost: \
1700 expected 4, got {}",
1701 id_of(&raced).get()
1702 )));
1703 }
1704 assert_payload(&raced, &json!({ "index": 4 }))
1705 }
1706
1707 /// A parked stream is woken by a push from another task.
1708 ///
1709 /// The single-poll tests above cannot prove this, and the reason is the one
1710 /// that bit T10: a future that never registers its waker returns `Pending`
1711 /// on its first poll *identically* to a working one. A harness that polls
1712 /// once and checks the answer distinguishes neither. So this runs on a real
1713 /// scheduler, with the push genuinely happening while the stream is parked
1714 /// and the assertion being that the parked task was actually resumed.
1715 #[tokio::test]
1716 async fn a_parked_stream_wakes_when_a_frame_is_pushed() -> Result<(), TestError> {
1717 let tail = tail(8)?;
1718 let planned = tail.plan(None);
1719 let mut stream = planned
1720 .try_stream(Duration::from_mins(15))
1721 .map_err(|_| Failure::UnexpectedStream)?;
1722
1723 // Nothing has been published, so the stream has nothing to say. The
1724 // tail is cloned rather than borrowed so the spawned task owns it —
1725 // a `spawn` future must be `'static`, and a borrow of the local would
1726 // not outlive this test.
1727 let publisher = tail.clone();
1728 let pusher = tokio::spawn(async move {
1729 // Yield first so the reader is genuinely parked when this lands,
1730 // rather than racing it and passing by luck.
1731 tokio::task::yield_now().await;
1732 publisher.push(json!({ "woken": true }));
1733 });
1734
1735 // The reader awaits the frame. This only completes if the waker was
1736 // registered before the push and honoured after it.
1737 let frame =
1738 core::future::poll_fn(|context| core::pin::Pin::new(&mut stream).poll_next(context))
1739 .await
1740 .ok_or_else(|| Failure::Wrong("the stream ended without a frame".to_owned()))?
1741 .map_err(|error| Failure::Wrong(error.to_string()))?;
1742
1743 pusher
1744 .await
1745 .map_err(|error| Failure::Wrong(error.to_string()))?;
1746
1747 let text = String::from_utf8_lossy(&frame).into_owned();
1748 let event = event_of(&text)?;
1749 assert_payload(&event, &json!({ "woken": true }))
1750 }
1751
1752 // ---------------------------------------------------------------------
1753 // The subscription is released.
1754 // ---------------------------------------------------------------------
1755
1756 /// Dropping a parked stream removes its registration.
1757 ///
1758 /// A client that disconnects is the *normal* way a stream ends, and the
1759 /// only place that can be observed is the destructor's effect on the tail.
1760 /// A `Drop` that did nothing would leak a waker per disconnection, and the
1761 /// leak is invisible from the response — the client simply stops receiving
1762 /// and nothing anywhere reports an error.
1763 #[test]
1764 fn dropping_the_body_releases_the_subscription() -> Result<(), TestError> {
1765 let tail = tail(4)?;
1766 let planned = tail.plan(None);
1767 let mut stream = planned
1768 .try_stream(no_keep_alive())
1769 .map_err(|_| Failure::UnexpectedStream)?;
1770
1771 let counting = Arc::new(Counting::default());
1772 let waker = counting.waker();
1773
1774 if !matches!(poll_once(&mut stream, &waker), Poll::Pending) {
1775 return Err(Failure::Wrong("the stream did not park".to_owned()));
1776 }
1777 if !stream.is_subscribed() {
1778 return Err(Failure::Wrong(
1779 "a parked stream must hold a subscription".to_owned(),
1780 ));
1781 }
1782 if parked_streams(&tail) != 1 {
1783 return Err(Failure::Wrong(format!(
1784 "expected one parked stream, got {}",
1785 parked_streams(&tail)
1786 )));
1787 }
1788
1789 drop(stream);
1790
1791 if parked_streams(&tail) != 0 {
1792 return Err(Failure::Wrong(format!(
1793 "a dropped stream left {} subscription(s) behind",
1794 parked_streams(&tail)
1795 )));
1796 }
1797 Ok(())
1798 }
1799
1800 /// One stream's drop does not unregister another's.
1801 ///
1802 /// The reason registrations carry an id rather than being matched by
1803 /// [`Waker::will_wake`]: two streams served from one task share a waker, so
1804 /// a `will_wake` match would let the first drop silence the second. A live
1805 /// client that stops receiving frames with no error anywhere is exactly the
1806 /// failure this guards.
1807 #[test]
1808 fn one_streams_drop_does_not_silence_another() -> Result<(), TestError> {
1809 let tail = tail(4)?;
1810 let planned = tail.plan(None);
1811 let mut first = planned
1812 .try_stream(no_keep_alive())
1813 .map_err(|_| Failure::UnexpectedStream)?;
1814 let mut second = planned
1815 .try_stream(no_keep_alive())
1816 .map_err(|_| Failure::UnexpectedStream)?;
1817
1818 // Both parked on the *same* waker, as two streams in one task would be.
1819 let counting = Arc::new(Counting::default());
1820 let waker = counting.waker();
1821 for stream in [&mut first, &mut second] {
1822 if !matches!(poll_once(stream, &waker), Poll::Pending) {
1823 return Err(Failure::Wrong("a stream did not park".to_owned()));
1824 }
1825 }
1826 if parked_streams(&tail) != 2 {
1827 return Err(Failure::Wrong(format!(
1828 "expected two parked streams on one waker, got {}",
1829 parked_streams(&tail)
1830 )));
1831 }
1832
1833 drop(first);
1834
1835 if parked_streams(&tail) != 1 {
1836 return Err(Failure::Wrong(format!(
1837 "dropping one stream disturbed the other: {} left",
1838 parked_streams(&tail)
1839 )));
1840 }
1841 if !second.is_subscribed() {
1842 return Err(Failure::Wrong(
1843 "the surviving stream lost its subscription".to_owned(),
1844 ));
1845 }
1846 drop(second);
1847 Ok(())
1848 }
1849
1850 /// A push to a parked stream wakes exactly the wakers it registered.
1851 ///
1852 /// The deterministic half of the claim the scheduled test above proves end
1853 /// to end. A registry that appends wakers but never calls `wake_by_ref`
1854 /// passes every test that only inspects state — the entry is there, the
1855 /// subscription is there, `parked_streams` is right — and hangs a real
1856 /// client forever, because the only thing that would ever re-poll the body
1857 /// is the wake that is never sent.
1858 ///
1859 /// The count is one rather than "at least one" because a `push` that woke
1860 /// the same waker twice would be a stream scheduled twice per frame, and a
1861 /// registry that grew per push would turn a long-lived stream into a
1862 /// quadratic one.
1863 #[test]
1864 fn a_push_wakes_the_parked_streams_exactly_once_each() -> Result<(), TestError> {
1865 let tail = tail(4)?;
1866 let planned = tail.plan(None);
1867 let mut first = planned
1868 .try_stream(no_keep_alive())
1869 .map_err(|_| Failure::UnexpectedStream)?;
1870 let mut second = planned
1871 .try_stream(no_keep_alive())
1872 .map_err(|_| Failure::UnexpectedStream)?;
1873
1874 let counting = Arc::new(Counting::default());
1875 let waker = counting.waker();
1876 for stream in [&mut first, &mut second] {
1877 if !matches!(poll_once(stream, &waker), Poll::Pending) {
1878 return Err(Failure::Wrong("a stream did not park".to_owned()));
1879 }
1880 }
1881 if counting.wakes() != 0 {
1882 return Err(Failure::Wrong(format!(
1883 "parking should not have woken anything, got {}",
1884 counting.wakes()
1885 )));
1886 }
1887
1888 tail.push(json!({ "index": 1 }));
1889
1890 // Both streams share one waker here, as two streams in one task would.
1891 // The registry holds two entries that both point at it, so one push
1892 // legitimately produces two wakes — the claim is that it is exactly two
1893 // and not one per poll, and not a wake per entry accumulated by the
1894 // parking itself.
1895 if counting.wakes() != 2 {
1896 return Err(Failure::Wrong(format!(
1897 "one push to two parked streams should wake twice, got {}",
1898 counting.wakes()
1899 )));
1900 }
1901
1902 // And the frames are actually there when the woken poll runs.
1903 let event = event_of(&bytes_of(poll_once(&mut first, &waker))?)?;
1904 assert_payload(&event, &json!({ "index": 1 }))
1905 }
1906
1907 // ---------------------------------------------------------------------
1908 // The keep-alive is a comment, not an event.
1909 // ---------------------------------------------------------------------
1910
1911 /// The keep-alive dispatches nothing, by the standard's own rule.
1912 ///
1913 /// Asserted through [`SseEvent::dispatch`] rather than by matching the
1914 /// bytes, because the claim is about what a *conforming client* does with
1915 /// the frame. A writer that emitted `data: \n\n` instead would pass a test
1916 /// that only checked for a leading colon, and every client would receive an
1917 /// empty message on every heartbeat.
1918 #[test]
1919 fn the_keep_alive_is_a_comment_the_client_dispatches_nothing_for() -> Result<(), TestError> {
1920 if KEEP_ALIVE_FRAME.lines().next() != Some(":ping") {
1921 return Err(Failure::Wrong(format!(
1922 "a keep-alive must be a comment line, got {KEEP_ALIVE_FRAME:?}"
1923 )));
1924 }
1925 if let Some(event) = SseEvent::dispatch(KEEP_ALIVE_FRAME)? {
1926 return Err(Failure::Wrong(format!(
1927 "a keep-alive dispatched an event: {}",
1928 event.data()
1929 )));
1930 }
1931 Ok(())
1932 }
1933
1934 /// The keep-alive fires on its interval and is the only thing an idle
1935 /// stream ever writes.
1936 ///
1937 /// A paused clock makes the interval part of the test instantaneous and
1938 /// deterministic. The claim is that a stream with nothing to say says
1939 /// *something* within the interval — without it a proxy between here and
1940 /// the client is free to decide an idle connection is dead, commonly at
1941 /// 60s, and close it while the server still believes the stream is
1942 /// healthy.
1943 #[tokio::test(start_paused = true)]
1944 async fn the_keep_alive_fires_after_its_interval_and_nothing_else() -> Result<(), TestError> {
1945 let tail = tail(4)?;
1946 let planned = tail.plan(None);
1947 let mut stream = planned
1948 .try_stream(Duration::from_secs(15))
1949 .map_err(|_| Failure::UnexpectedStream)?;
1950
1951 let counting = Arc::new(Counting::default());
1952 let waker = counting.waker();
1953
1954 // Before the interval, silence.
1955 if !matches!(poll_once(&mut stream, &waker), Poll::Pending) {
1956 return Err(Failure::Wrong(
1957 "an idle stream wrote before its keep-alive interval elapsed".to_owned(),
1958 ));
1959 }
1960
1961 tokio::time::advance(Duration::from_secs(16)).await;
1962
1963 let frame = bytes_of(poll_once(&mut stream, &waker))?;
1964 if frame != KEEP_ALIVE_FRAME {
1965 return Err(Failure::Wrong(format!(
1966 "expected the keep-alive comment, got {frame:?}"
1967 )));
1968 }
1969 // And it is a comment, not an event, on the wire and not only in the
1970 // constant.
1971 if SseEvent::dispatch(&frame)?.is_some() {
1972 return Err(Failure::Wrong(
1973 "the emitted keep-alive dispatched an event".to_owned(),
1974 ));
1975 }
1976 Ok(())
1977 }
1978
1979 // ---------------------------------------------------------------------
1980 // The response shape.
1981 // ---------------------------------------------------------------------
1982
1983 /// A stream response carries the three headers a proxy needs.
1984 ///
1985 /// `Content-Type` is what makes a client parse the bytes as a stream at
1986 /// all. `Cache-Control: no-cache` is what stops an intermediary serving a
1987 /// stored response to the *next* client, which for a stream means that
1988 /// client receives the previous one's events. `X-Accel-Buffering: no` is
1989 /// nginx's opt-out, and without it nginx buffers and delivers in lumps —
1990 /// which presents as a live server that has gone silent and is the commonest
1991 /// way a correct SSE server looks broken.
1992 #[tokio::test]
1993 async fn the_stream_response_carries_the_headers_a_proxy_needs() -> Result<(), TestError> {
1994 let tail = tail(4)?;
1995 let response = tail
1996 .respond(None, no_keep_alive())
1997 .map_err(|error| Failure::Wrong(error.to_string()))?;
1998
1999 if response.status() != StatusCode::OK {
2000 return Err(Failure::Wrong(format!(
2001 "a resumable stream is a 200, got {}",
2002 response.status()
2003 )));
2004 }
2005 let headers = response.headers();
2006 let content_type = headers
2007 .get(axum::http::header::CONTENT_TYPE)
2008 .and_then(|value| value.to_str().ok())
2009 .unwrap_or_default();
2010 if content_type != super::EVENT_STREAM {
2011 return Err(Failure::Wrong(format!(
2012 "expected {}, got {content_type:?}",
2013 super::EVENT_STREAM
2014 )));
2015 }
2016 let cache_control = headers
2017 .get(axum::http::header::CACHE_CONTROL)
2018 .and_then(|value| value.to_str().ok())
2019 .unwrap_or_default();
2020 if cache_control != "no-cache" {
2021 return Err(Failure::Wrong(format!(
2022 "expected no-cache, got {cache_control:?}"
2023 )));
2024 }
2025 let buffering = headers
2026 .get(super::ACCEL_BUFFERING)
2027 .and_then(|value| value.to_str().ok())
2028 .unwrap_or_default();
2029 if buffering != "no" {
2030 return Err(Failure::Wrong(format!(
2031 "expected the proxy buffering opt-out, got {buffering:?}"
2032 )));
2033 }
2034 Ok(())
2035 }
2036
2037 /// The response body carries the frames the replay queued.
2038 ///
2039 /// The one test that goes through the real [`Body`] rather than driving the
2040 /// stream by hand, because the header set and the body are assembled in
2041 /// different places and either could be wrong independently.
2042 ///
2043 /// It reads one frame rather than draining, and the read is bounded by a
2044 /// timeout. `to_bytes` would never return: the body is a live stream and
2045 /// does not end, so a test waiting for an end hangs rather than fails. The
2046 /// timeout makes the claim precise — the first frame arrives promptly — and
2047 /// a stream that delivered nothing is a failure, not an absence of one.
2048 #[tokio::test]
2049 async fn the_response_body_carries_the_frames_the_replay_queued() -> Result<(), TestError> {
2050 let tail = tail(8)?;
2051 for index in 1..=2 {
2052 tail.push(json!({ "index": index }));
2053 }
2054 let response = tail
2055 .respond(Some("0"), no_keep_alive())
2056 .map_err(|error| Failure::Wrong(error.to_string()))?;
2057 let mut chunks = response.into_body().into_data_stream();
2058
2059 let first = tokio::time::timeout(
2060 Duration::from_secs(5),
2061 core::future::poll_fn(|context| Pin::new(&mut chunks).poll_next(context)),
2062 )
2063 .await
2064 .map_err(|_| Failure::Wrong("the body never produced a chunk".to_owned()))?
2065 .ok_or_else(|| Failure::Wrong("the body ended without a chunk".to_owned()))?
2066 .map_err(|error| Failure::Wrong(error.to_string()))?;
2067
2068 let text = String::from_utf8_lossy(&first).into_owned();
2069 // A chunk boundary can fall inside a frame, so this asserts only that
2070 // the body starts with the first replayed frame. The ordering claims
2071 // live in the tests above, which drive the stream frame by frame.
2072 if !text.starts_with("id: 1\n") {
2073 return Err(Failure::Wrong(format!(
2074 "expected the body to start with frame 1, got {text:?}"
2075 )));
2076 }
2077 Ok(())
2078 }
2079
2080 /// A `Planned` refusal is a response, and the same decision is not a
2081 /// stream at the same time.
2082 ///
2083 /// The type-level claim, stated once: `try_stream` returns
2084 /// `Result<EventStream, Response>`, so a caller that wants the body is
2085 /// forced by the compiler to handle the refusal. A helper that returned
2086 /// `EventStream` and put the status somewhere else would make "a 200 with
2087 /// an error inside it" a one-line mistake.
2088 #[test]
2089 fn a_refusal_has_a_response_and_not_a_stream() -> Result<(), TestError> {
2090 let tail = tail(4)?;
2091 let planned = tail.plan(Some("garbage"));
2092 if planned.refusal().is_none() {
2093 return Err(Failure::Wrong(
2094 "an unreadable cursor should have been refused".to_owned(),
2095 ));
2096 }
2097 // The refusal is the `Err`, not a `None`, and it is a `Response` — so
2098 // the caller that wants the body is forced by the compiler to handle
2099 // the refusal rather than receiving an empty stream it will serve as a
2100 // 200.
2101 match planned.try_stream(no_keep_alive()) {
2102 Ok(_) => Err(Failure::Wrong("a refusal produced a stream".to_owned())),
2103 Err(response) => {
2104 if response.status().is_success() {
2105 return Err(Failure::Wrong(
2106 "a refusal produced a success status".to_owned(),
2107 ));
2108 }
2109 Ok(())
2110 }
2111 }
2112 }
2113
2114 /// The `Resume` a plan made is the one the stream starts from.
2115 ///
2116 /// A thin invariant, but it is the one that makes the rest of the file
2117 /// legible: the frames queued at construction are exactly
2118 /// [`Resume::pending`], so a test reading a plan and a test reading the
2119 /// stream cannot disagree about what was decided.
2120 #[test]
2121 fn the_queued_frames_are_exactly_the_decisions_pending_frames() -> Result<(), TestError> {
2122 let source = tail(4)?;
2123 source.push(json!({ "index": 1 }));
2124 let planned = source.plan(Some("0"));
2125 let expected = planned.resume().pending().len();
2126 if expected != 1 {
2127 return Err(Failure::Wrong(format!(
2128 "expected one pending frame for a fresh client, got {expected}"
2129 )));
2130 }
2131 // A `Gap` plan queues exactly one frame — the notice — and a `Live`
2132 // queues none. Both are properties of `resume`, but the queue is this
2133 // module's, so the claim is asserted here too.
2134 let evicting = tail(1)?;
2135 evicting.push(json!({ "index": 1 }));
2136 evicting.push(json!({ "index": 2 }));
2137 let behind = evicting.plan(Some("0"));
2138 if !matches!(behind.resume(), Resume::Gap { .. }) {
2139 return Err(Failure::Wrong(
2140 "a client behind the eviction point should be a gap".to_owned(),
2141 ));
2142 }
2143 Ok(())
2144 }
2145}