Skip to main content

windows_file_watcher/
queue.rs

1// Copyright (c) 2026 Mike Grier
2//! The crate-owned notification queue: where a watcher puts what it decodes, and
3//! where a client takes it from.
4//!
5//! The queue is the boundary that keeps client behaviour off this crate's cadence
6//! path. Delivery is an enqueue *the crate performs*; the client only ever
7//! receives. There is no sink trait and no client-supplied closure, so nothing a
8//! client does -- blocking, panicking, being slow -- can stall or unwind a
9//! completion callback (D-2/D-11).
10//!
11//! Note this is a statement about the **call graph**, not about threads. Which
12//! thread a client drains on is entirely its own business, and draining from its
13//! own thread-pool callback is an expected integration; that is the client's pool
14//! object and the client's cadence. The M3.4 doorbell (D-25) exists to make
15//! precisely that integration possible without dedicating a thread.
16//!
17//! # Reserved is guaranteed, unreserved is best-effort
18//!
19//! The queue is bounded, so it can fill, and the two kinds of thing that travel
20//! it cannot survive that the same way. A lost change notification is recoverable
21//! -- re-scanning is exactly what [`DesyncCause`] exists to ask for -- while a
22//! lost completion or fault report is a liveness bug, because the client waits
23//! forever for something that already happened.
24//!
25//! Rather than sort that out per message type at the enqueue, reliability is a
26//! property of **reserved capacity** (D-33). A control message takes its slot
27//! through [`Sender::reserve`] *before* whatever produces it is allowed to
28//! proceed, so [`Reservation::send`] cannot fail -- the room is already the
29//! sender's. Change notifications reserve nothing and go through
30//! [`Sender::send`], which is allowed to fail. One line covers both: *reserved is
31//! guaranteed, unreserved is best-effort.*
32//!
33//! # Reporting a full queue without needing room to do it
34//!
35//! A desync is an ordinary notification and rides the queue in order like any
36//! other (D-12/D-26), which is what lets a client say "everything before this is
37//! accounted for". That breaks down in exactly one place: reporting that the
38//! queue is full cannot itself require a slot in the full queue, and reserving
39//! one only defers the problem to the second overflow.
40//!
41//! So a failed enqueue latches the affected [`WatchId`] in a set held *outside*
42//! the bounded queue, where it coalesces -- lossless, because a desync is
43//! idempotent and the answer is always a re-scan. The latch is drained back into
44//! the queue at the next successful enqueue, which is also the position the loss
45//! actually occupies: the queue was full when the loss happened, so everything
46//! still queued precedes it and everything enqueued afterwards follows it.
47//! A receiver that reaches an empty queue synthesises any remainder directly, so
48//! a latched desync is delivered even if nothing further is ever sent.
49//!
50//! # The doorbell
51//!
52//! A queue drainable only by a blocking [`Receiver::recv`] would force a client
53//! that already owns a thread pool to dedicate a thread to it -- contradicting
54//! this crate's premise that nobody should have to own threads. So the receiver
55//! hands out a manual-reset event ([`Receiver::doorbell`]) that a client can wait
56//! on however it likes, including from its own `ThreadpoolWait`.
57//!
58//! It is created **lazily**, so a client that only ever calls `recv` allocates no
59//! kernel object at all.
60//!
61//! *Why not a `Doorbell` trait the client implements?* Because that would be a
62//! client callback on this crate's cadence path, which is the one thing D-2
63//! forbids, and it would have made `Monitor`, `Session`, and `Sender` all generic
64//! over it. The composition argument for a trait does not survive contact with
65//! the platform either: on Windows a HANDLE **is** the universal waitable
66//! currency -- `WaitForSingleObject`, `WaitForMultipleObjects`,
67//! `MsgWaitForMultipleObjects`, `ThreadpoolWait` and alertable waits all take one
68//! -- so an event is the native composition point rather than a lowest common
69//! denominator. The single case it does not reach, an async `Waker`, is a short
70//! bridge the client writes on its own pool, which is where that code belongs.
71//! Owning the doorbell also makes the reset discipline an internal invariant
72//! rather than a client obligation (D-25).
73//!
74//! That invariant is one line: **the event is signalled exactly when the receiver
75//! has something to observe**, which is a queued notification, an owed loss
76//! report, or the end of the stream. It is re-established under the queue lock at
77//! the end of every mutation -- the same lock a receiver holds while deciding
78//! there is nothing to take -- so a wakeup cannot be lost in the gap between
79//! those two decisions, because there is no gap.
80//!
81//! This is the crate-owned notification queue. A client holds the [`Receiver`]
82//! and drains it; nothing in this module is reachable from client code except
83//! through that.
84
85use std::collections::VecDeque;
86use std::io;
87use std::num::NonZeroUsize;
88use std::os::windows::io::{AsHandle, AsRawHandle, BorrowedHandle, FromRawHandle, OwnedHandle};
89use std::ptr;
90use std::sync::{Arc, Condvar, Mutex, OnceLock, Weak};
91use std::time::{Duration, Instant};
92
93use win_handles_sys::{DuplicateAccess, Inheritance, duplicate_handle};
94use win_shared_os_owned_handle::SharedHandle;
95use windows_sys::Win32::Foundation::{FALSE, TRUE};
96use windows_sys::Win32::System::Threading::{CreateEventW, ResetEvent, SetEvent};
97
98use crate::directory::{FaultDetail, VolumeIdentity};
99use crate::notify::{Change, DesyncCause};
100use crate::retry::{FaultOperation, WatchMode};
101
102/// The bound used when a caller does not choose one.
103///
104/// Counts **notifications**, not changes: one decoded completion is one batch
105/// (D-10) and a batch can carry hundreds of records, so this is a far deeper
106/// backlog than the number suggests. It is chosen to be ample for a client that
107/// drains promptly and still bounded for one that stops.
108pub const DEFAULT_BOUND: NonZeroUsize = match NonZeroUsize::new(1024) {
109    Some(bound) => bound,
110    None => unreachable!(),
111};
112
113/// Identifies the subscription a notification belongs to.
114///
115/// `Copy`, so a client can retain it to route or aggregate without holding the
116/// subscription's lifecycle object (D-5). M3.4 issues these from the monitor;
117/// until then a watcher is constructed with one.
118#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)]
119pub struct WatchId(u64);
120
121impl WatchId {
122    /// Build an identifier from a raw value.
123    ///
124    /// In production the monitor issues `WatchId`s. This constructor lets a
125    /// downstream consumer mint one to tag a synthetic [`Notification`] when
126    /// testing its own handler against a `Receiver` it drives (D-81); any value
127    /// is a valid identifier, so the pairing between an id and its notifications
128    /// is the consumer's to choose.
129    #[must_use]
130    pub fn from_raw(value: u64) -> Self {
131        Self(value)
132    }
133
134    /// The raw value, for a client that wants to key a map on it.
135    #[must_use]
136    pub fn get(self) -> u64 {
137        self.0
138    }
139}
140
141/// How a request turned out.
142///
143/// Registration is asynchronous (D-2), so the outcome cannot be the return value
144/// of the call that made the request; it arrives here instead.
145#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
146pub enum Outcome {
147    /// The subscription is registered and the monitor is watching for it.
148    Subscribed,
149    /// The subscription is registered, but the target could not be opened yet.
150    ///
151    /// Retryable in D-22's sense -- a path that does not exist yet, a network
152    /// share that is briefly unreachable -- so the monitor keeps the
153    /// subscription and will establish it when it can. This is not a failure and
154    /// carries no terminal meaning (D-14).
155    Establishing,
156    /// The target can never be watched, so nothing was registered.
157    ///
158    /// Only D-22's permanent pair reaches here, and both mean the caller named
159    /// something that is not a watchable directory rather than that anything went
160    /// wrong in the environment. Retrying would spin forever against input that
161    /// will never become valid.
162    Failed {
163        /// The classification and raw code behind the permanent failure (D-79).
164        detail: FaultDetail,
165    },
166    /// The subscription has ended and its watcher is released.
167    ///
168    /// Its position in the stream is the guarantee: everything before it belongs
169    /// to the live watch, and nothing after it does (D-30).
170    Cancelled,
171}
172
173/// One item a client receives, always tagged with the subscription it belongs to.
174///
175/// Changes and desyncs ride the same queue so their order relative to one another
176/// is well defined within a subscription (D-12): a client that sees a `Desync`
177/// knows every change enqueued before it, and none after, is accounted for.
178#[derive(Clone, Debug, PartialEq, Eq)]
179pub enum Notification {
180    /// The changes one completion carried, in the order the kernel reported them.
181    Batch {
182        /// The subscription these changes belong to.
183        watch: WatchId,
184        /// The changes, in kernel order.
185        changes: Vec<Change>,
186    },
187    /// Changes were lost; the client should re-scan (D-12).
188    Desync {
189        /// The subscription affected.
190        watch: WatchId,
191        /// How the gap arose. Advisory for the four recoverable causes -- the
192        /// response is a re-scan either way -- but
193        /// [`DesyncCause::Stopped`] is terminal and must be matched on.
194        cause: DesyncCause,
195    },
196    /// A request the client made has been serviced (D-30).
197    ///
198    /// Carried on this queue rather than a side channel so that its ordering
199    /// against data is **structural** rather than temporal.
200    Completion {
201        /// The subscription the request concerned.
202        watch: WatchId,
203        /// What happened.
204        outcome: Outcome,
205    },
206    /// The watch stopped delivering while the monitor works to re-establish it
207    /// (D-14/D-15/D-31). Opt-in via `WatchOptions::report_liveness` (D-13).
208    Suspended {
209        /// The subscription affected.
210        watch: WatchId,
211    },
212    /// The watch is delivering again after a [`Notification::Suspended`]. A
213    /// [`Notification::Desync { Reestablished }`](DesyncCause::Reestablished)
214    /// always precedes or accompanies this, since the gap it bridges may have
215    /// hidden changes. Opt-in (D-13).
216    Resumed {
217        /// The subscription affected.
218        watch: WatchId,
219    },
220    /// Which tier is actually watching this subscription's target, reported
221    /// once at first establishment and again after every re-establishment:
222    /// [`WatchMode::Detailed`] or the [`WatchMode::Coarse`] fallback (D-17).
223    /// Opt-in (D-13).
224    Established {
225        /// The subscription affected.
226        watch: WatchId,
227        /// The tier now watching.
228        mode: WatchMode,
229    },
230    /// The monitor is asking this **interactive** subscription (D-27) how long
231    /// to wait before the next attempt. Answer through
232    /// [`crate::session::Session::answer`]; an explicit `answer(watch, None)`
233    /// declines, which is counted at the operation's default delay -- never
234    /// answering at all leaves the question outstanding indefinitely. This is
235    /// the one message a fault protocol cannot afford to lose, so it rides a
236    /// standing reservation (`Sender::reserve_standing`) taken once at
237    /// registration rather than the best-effort path -- sound because a watcher
238    /// cannot fault twice concurrently (D-28), so at most one question per
239    /// subscription is ever outstanding.
240    RetryQuestion {
241        /// The subscription being asked.
242        watch: WatchId,
243        /// Which operation faulted.
244        operation: FaultOperation,
245        /// The classification and raw code behind the failure (D-79).
246        detail: FaultDetail,
247    },
248    /// A path-based reopen landed on a different volume than this
249    /// subscription started on, and it opted into
250    /// [`crate::VolumeChangePolicy::Confirm`] (D-78/M12). Answer through
251    /// [`crate::session::Session::answer_volume_change`];
252    /// [`crate::VolumeChangeDecision::Stop`] removes just this subscription,
253    /// [`crate::VolumeChangeDecision::Continue`] keeps it running against the
254    /// new volume. Rides the same standing reservation `RetryQuestion` does
255    /// (never outstanding at the same time for one subscription, D-28).
256    VolumeChanged {
257        /// The subscription being asked.
258        watch: WatchId,
259        /// The volume this subscription was previously confirmed on.
260        previous: VolumeIdentity,
261        /// The volume the reopen just landed on.
262        current: VolumeIdentity,
263    },
264}
265
266impl Notification {
267    /// The subscription this notification belongs to.
268    #[must_use]
269    pub fn watch(&self) -> WatchId {
270        match self {
271            Notification::Batch { watch, .. }
272            | Notification::Desync { watch, .. }
273            | Notification::Completion { watch, .. }
274            | Notification::Suspended { watch }
275            | Notification::Resumed { watch }
276            | Notification::Established { watch, .. }
277            | Notification::RetryQuestion { watch, .. }
278            | Notification::VolumeChanged { watch, .. } => *watch,
279        }
280    }
281}
282
283/// The shared queue storage.
284struct Shared {
285    items: Mutex<State>,
286    arrived: Condvar,
287    /// The client-facing wake, created on first request and never replaced.
288    ///
289    /// Outside the lock so a borrow of it can outlive the guard, but only ever
290    /// signalled or reset *under* the lock -- see [`refresh_doorbell`].
291    doorbell: OnceLock<SharedHandle>,
292}
293
294/// A crate-owned producer that stopped because the queue was full, and must be
295/// prodded when there is room again.
296///
297/// This is **not** the client callback D-2 forbids and D-25 rejected: every
298/// implementor is this crate's own code, and the contract is one line -- return
299/// promptly, touch nothing of the queue's. Observation is the only tier that
300/// needs it, because it is the only one that reserves nothing (D-33).
301pub(crate) trait Resume: Send + Sync {
302    /// There may be room now. Called with no lock of the queue's held.
303    fn resume(&self);
304}
305
306/// One queued notification, optionally carrying a [`StandingSlot`]'s permanent
307/// reservation for as long as it sits in the queue.
308///
309/// A `StandingSlot` send transfers its one carved-out unit of `reserved`
310/// capacity into the queue for as long as the message sits there (so `free()`
311/// is never double-charged for it, once as `reserved` and again as an
312/// occupied queue slot). [`take`] hands that unit back, inline with the pop
313/// that removes this entry; the [`StandingHold`] is the record that the
314/// transfer happened, which is what lets [`StandingSlot::drop`] tell whether it
315/// still owns that unit itself (no hold outstanding) or has already
316/// transferred it to a queued entry that has not been drained yet.
317struct Entry {
318    notification: EntryNotification,
319    _standing_hold: Option<StandingHold>,
320}
321
322/// A plain entry's notification is fixed at push time; a standing entry's
323/// lives in its paired [`StandingHold`]'s [`StandingState::queued`] instead,
324/// so a second [`StandingSlot::send`] while this entry is still undrained can
325/// coalesce into it in place (PR #20 review response) rather than pushing a
326/// second entry and double-spending the slot's single unit of `reserved`.
327enum EntryNotification {
328    Plain(Notification),
329    Standing,
330}
331
332impl Entry {
333    fn plain(notification: Notification) -> Self {
334        Self {
335            notification: EntryNotification::Plain(notification),
336            _standing_hold: None,
337        }
338    }
339
340    fn standing(hold: StandingHold) -> Self {
341        Self {
342            notification: EntryNotification::Standing,
343            _standing_hold: Some(hold),
344        }
345    }
346
347    /// The notification this entry carries, reading a standing entry's
348    /// current (possibly coalesced) value from its `StandingHold` before the
349    /// entry itself is discarded.
350    fn into_notification(self) -> Notification {
351        match self.notification {
352            EntryNotification::Plain(notification) => notification,
353            EntryNotification::Standing => {
354                let hold = self
355                    ._standing_hold
356                    .as_ref()
357                    .expect("a Standing entry always carries a StandingHold");
358                lock(&hold.state)
359                    .queued
360                    .clone()
361                    .expect("a Standing entry's queued value is set for as long as it is undrained")
362            }
363        }
364    }
365}
366
367struct State {
368    queue: VecDeque<Entry>,
369    /// The bound. Never zero, which is what makes the delivery guarantee
370    /// non-vacuous: a queue with no room could never carry the desync that
371    /// reports its own saturation.
372    capacity: usize,
373    /// Slots taken by an outstanding [`Reservation`] but not yet filled. Held
374    /// away from the best-effort path, which is the whole mechanism: a reserved
375    /// send cannot fail because nothing else can take its room.
376    reserved: usize,
377    /// Subscriptions with a loss that could not be enqueued, in the order they
378    /// were first latched.
379    ///
380    /// Deliberately outside the bounded queue, and deliberately coalesced: a
381    /// second loss for a subscription that already owes a desync adds nothing,
382    /// because the client's response to one is the same as its response to ten.
383    latched: VecDeque<WatchId>,
384    /// Set when every sender is gone, so a blocked receiver can stop waiting
385    /// rather than hang forever on a queue nothing can fill.
386    senders: usize,
387    /// Producers to prod when a saturated queue frees a slot.
388    ///
389    /// Weak, so a producer that has gone away is simply forgotten rather than
390    /// kept alive by the queue it fed.
391    resumers: Vec<Weak<dyn Resume>>,
392}
393
394impl State {
395    /// Slots available to the best-effort path: neither occupied nor reserved.
396    fn free(&self) -> usize {
397        self.capacity - self.queue.len() - self.reserved
398    }
399
400    /// Slots a best-effort notification could actually take *right now*.
401    ///
402    /// [`Sender::send`] flushes every owed latch into the queue before it
403    /// considers the caller's own notification, so a free slot with a latch
404    /// outstanding already belongs to that flush. This is therefore
405    /// `free()` minus what is owed, and it is the **single** definition both
406    /// [`Sender::has_room`] (`> 0`) and [`freed_resumers`]' wake edge (`== 1`)
407    /// are expressed in terms of.
408    ///
409    /// Keeping them derived from one quantity is deliberate: they were
410    /// previously two independent expressions, `free() > latched.len()` and
411    /// `free() == 1`, which disagreed the moment anything was latched -- the
412    /// prod fired one slot too early, the re-check still said "no room", and no
413    /// later drain re-fired it, wedging the watcher permanently
414    /// (PR #42 review). A predicate and the edge that wakes it cannot be
415    /// allowed to drift apart.
416    fn best_effort_room(&self) -> usize {
417        self.free().saturating_sub(self.latched.len())
418    }
419
420    /// Whether a receiver has anything to observe.
421    ///
422    /// Disconnection counts: a client waiting on the doorbell must learn that the
423    /// stream has ended, or it would wait for a notification nothing can send.
424    fn pending(&self) -> bool {
425        !self.queue.is_empty() || !self.latched.is_empty() || self.senders == 0
426    }
427
428    /// Move latched losses back into the queue, as many as there is room for.
429    ///
430    /// Called immediately before an enqueue, which is what puts each desync in
431    /// the position the loss actually occupies -- see the module docs.
432    fn flush_latched(&mut self) {
433        while self.free() > 0 {
434            let Some(watch) = self.latched.pop_front() else {
435                break;
436            };
437            self.queue.push_back(Entry::plain(Notification::Desync {
438                watch,
439                cause: DesyncCause::QueueFull,
440            }));
441        }
442    }
443
444    /// Record that a subscription lost something, coalescing with any loss it is
445    /// already owed.
446    fn latch(&mut self, watch: WatchId) {
447        if !self.latched.contains(&watch) {
448            self.latched.push_back(watch);
449        }
450    }
451}
452
453/// What became of a best-effort notification.
454#[derive(Clone, Copy, Debug, PartialEq, Eq)]
455#[must_use = "a latched notification was lost, which M3.7 responds to by not re-arming"]
456pub enum Delivery {
457    /// Enqueued. The client will receive it.
458    Queued,
459    /// The queue was full, so it was dropped and a `Desync { QueueFull }` is now
460    /// owed to its subscription.
461    Latched,
462}
463
464/// The crate-side half: enqueues, never blocks.
465///
466/// Cloneable and `Send + Sync` because several watchers -- whose completions run
467/// on different pool threads -- feed one client queue (D-11).
468pub struct Sender {
469    shared: Arc<Shared>,
470}
471
472impl Clone for Sender {
473    fn clone(&self) -> Self {
474        let mut state = lock(&self.shared.items);
475        state.senders += 1;
476        drop(state);
477        Self {
478            shared: Arc::clone(&self.shared),
479        }
480    }
481}
482
483impl Sender {
484    /// Enqueue one notification, best-effort.
485    ///
486    /// Runs on the cadence path, so it never blocks. It reserves nothing, so it
487    /// *can* fail: a full queue drops the notification and latches a
488    /// `Desync { QueueFull }` against its subscription (D-29/D-33). Use
489    /// [`Sender::reserve`] for anything whose loss a client cannot recover from.
490    ///
491    /// A downstream consumer may call this on a `Sender` from
492    /// [`channel_with_bound`] to feed its own handler synthetic notifications in
493    /// a test (behind the `test-util` feature); the returned [`Delivery`] reports
494    /// whether the queue accepted the notification or dropped-and-latched it,
495    /// exactly as in production.
496    pub fn send(&self, notification: Notification) -> Delivery {
497        let watch = notification.watch();
498        let delivery = {
499            let mut state = lock(&self.shared.items);
500            // Before the new item, so a loss is reported at the point it happened
501            // rather than after changes that preceded it.
502            state.flush_latched();
503            let delivery = if state.free() > 0 {
504                state.queue.push_back(Entry::plain(notification));
505                Delivery::Queued
506            } else {
507                state.latch(watch);
508                Delivery::Latched
509            };
510            refresh_doorbell(&self.shared, &state);
511            delivery
512        };
513        self.shared.arrived.notify_all();
514        delivery
515    }
516
517    /// Claim one slot for a message that must not be lost.
518    ///
519    /// Take this *before* starting whatever produces the message (D-33): a
520    /// request reserves its completion slot at submit, an interactive
521    /// subscription its fault slot at registration. Backpressure then lands here,
522    /// on the caller's own thread, where it can be handled -- rather than at a
523    /// delivery that has no way to fail safely.
524    ///
525    /// Returns `None` if the queue is full. Dropping the reservation without
526    /// sending releases the slot.
527    pub fn reserve(&self) -> Option<Reservation> {
528        {
529            let mut state = lock(&self.shared.items);
530            if state.free() == 0 {
531                return None;
532            }
533            state.reserved += 1;
534        }
535        // Cloning takes the same lock, so it happens after the guard above is
536        // released. The clone is what keeps the queue connected for as long as
537        // the reservation is outstanding: a reservation can still deliver, so a
538        // receiver must not see the stream as finished while one exists.
539        Some(Reservation {
540            sender: self.clone(),
541            used: false,
542        })
543    }
544
545    /// Permanently reserve one slot for this subscription's fault question
546    /// (D-27/D-28).
547    ///
548    /// Unlike [`Sender::reserve`], the capacity this claims is never returned:
549    /// the same subscription may fault many times over its life, and
550    /// [`StandingSlot::send`] must never be able to fail. That is sound because
551    /// a watcher cannot be faulted twice concurrently (D-28) -- the state
552    /// machine cannot ask a second question before the first is answered -- so
553    /// one permanently-reserved slot is always enough, unlike
554    /// [`Sender::reserve`]'s slot, which is claimed and released per message.
555    ///
556    /// Returns `None` if the queue has no free capacity to carve out at
557    /// registration.
558    pub(crate) fn reserve_standing(&self) -> Option<StandingSlot> {
559        {
560            let mut state = lock(&self.shared.items);
561            if state.free() == 0 {
562                return None;
563            }
564            state.reserved += 1;
565        }
566        Some(StandingSlot {
567            sender: self.clone(),
568            state: Arc::new(Mutex::new(StandingState {
569                slot_alive: true,
570                in_flight: false,
571                queued: None,
572            })),
573        })
574    }
575
576    /// Whether a best-effort notification would be accepted right now.
577    ///
578    /// The observation tier checks this *before* arming a read rather than
579    /// discovering it at the enqueue, which is what turns saturation into a
580    /// grace period in the kernel's own change buffer instead of a loss (D-29).
581    ///
582    /// Accounts for a pending latched loss: [`Sender::send`] always flushes
583    /// every owed [`Delivery::Latched`] desync into the queue *before*
584    /// considering the caller's own notification, so a freed slot with a
585    /// latch still outstanding is not actually available to it. Without this,
586    /// a caller could see `true` immediately after a slot freed up, then
587    /// still get `Delivery::Latched` back from the very next `send` -- the
588    /// flush consumed the slot this check saw.
589    pub fn has_room(&self) -> bool {
590        lock(&self.shared.items).best_effort_room() > 0
591    }
592
593    /// Ask to be prodded when a saturated queue frees a slot.
594    ///
595    /// Registering is idempotent in effect: a spurious prod costs a producer only
596    /// a re-check.
597    pub(crate) fn register_resume(&self, who: &Arc<impl Resume + 'static>) {
598        lock(&self.shared.items)
599            .resumers
600            .push(Arc::downgrade(who) as Weak<dyn Resume>);
601    }
602}
603
604/// A claimed slot in the notification queue.
605///
606/// Holding one is what makes delivery infallible, so a control message is
607/// produced only once its room is already secured (D-33). Releasing it without
608/// sending is fine -- the slot returns to the pool -- and is what happens when a
609/// request is abandoned before it completes.
610pub struct Reservation {
611    sender: Sender,
612    used: bool,
613}
614
615impl Reservation {
616    /// Deliver the notification into the reserved slot.
617    ///
618    /// Cannot fail: the room was taken when this reservation was made.
619    pub fn send(mut self, notification: Notification) {
620        {
621            let mut state = lock(&self.sender.shared.items);
622            // Flushed while this reservation is still counted, so the latch can
623            // only take genuinely free slots and never this one.
624            state.flush_latched();
625            state.reserved -= 1;
626            state.queue.push_back(Entry::plain(notification));
627            refresh_doorbell(&self.sender.shared, &state);
628        }
629        self.used = true;
630        self.sender.shared.arrived.notify_all();
631    }
632}
633
634impl Drop for Reservation {
635    fn drop(&mut self) {
636        if !self.used {
637            let resumers = {
638                let mut state = lock(&self.sender.shared.items);
639                state.reserved -= 1;
640                // A dropped-unused reservation frees capacity exactly like a
641                // drained item does, so a producer paused in `Backpressured`
642                // (D-29) must be prodded the same way, or it can stay paused
643                // forever waiting on a receive that was never going to come.
644                freed_resumers(&mut state, true)
645            };
646            prod(resumers);
647        }
648    }
649}
650
651/// State shared between a [`StandingSlot`] and every [`StandingHold`] it has
652/// ever handed to a queued entry, protected by its own lock so the two halves
653/// of a reservation's transfer -- whether the slot is still alive, and
654/// whether its last send is still queued -- are always read and mutated
655/// together with the `reserved` count they gate, never observed mid-way.
656///
657/// Lock order is fixed as *the queue's `items`, then this* everywhere it is
658/// taken (`send`, `StandingSlot::drop`, and [`take`]), which is what makes the
659/// combined transition atomic: whichever of `StandingSlot::drop` or `take` runs
660/// first holds this lock for its entire mutation of `reserved`, so the other
661/// side cannot observe a half-finished transfer. `StandingHold::drop` takes
662/// neither lock -- see that impl for why it must not.
663struct StandingState {
664    /// Whether the owning `StandingSlot` has been dropped. Once false, no
665    /// further send can ever happen, so an outstanding hold's eventual drain
666    /// must simply let its transferred unit go free rather than handing it
667    /// back to a slot that no longer exists to reuse it.
668    slot_alive: bool,
669    /// Whether the last-sent message is still queued, undrained -- meaning
670    /// the unit of `reserved` capacity this slot carved out is currently
671    /// manifested as that queued entry rather than sitting in `reserved`.
672    in_flight: bool,
673    /// The queued entry's current notification value while `in_flight` is
674    /// true (PR #20 review response); `None` otherwise. A second `send`
675    /// while `in_flight` overwrites this in place -- reachable when an
676    /// interactive watch is answered before its queued question is drained
677    /// and the retry fails again -- rather than decrementing `reserved` a
678    /// second time for a slot that only ever carves out one unit.
679    queued: Option<Notification>,
680}
681
682/// A permanent slot in the notification queue, reserved once and reused for as
683/// long as the subscription lives (D-27/D-28).
684///
685/// See [`Sender::reserve_standing`] for why one slot is always enough. Unlike
686/// [`Reservation`], sending through this does not release the reservation --
687/// there is nothing to release until the subscription itself ends, at which
688/// point `Drop` returns the slot to the pool.
689pub(crate) struct StandingSlot {
690    sender: Sender,
691    state: Arc<Mutex<StandingState>>,
692}
693
694impl StandingSlot {
695    /// Deliver the notification. Cannot fail: the slot was carved out of
696    /// capacity at registration and is never returned while this is held.
697    ///
698    /// Coalesces in place with a previous send whose entry is still queued
699    /// (undrained) rather than pushing a second entry (PR #20 review
700    /// response): the slot's single carved-out unit of `reserved` capacity
701    /// must never be double-spent, and it is exactly this second-send-while-
702    /// undrained case that would otherwise underflow it.
703    ///
704    /// Lock order: `items`, then this slot's own `standing` state -- the
705    /// same order [`take`] and [`StandingSlot::drop`] use, so none of the
706    /// three can ever deadlock waiting on each other.
707    pub(crate) fn send(&self, notification: Notification) {
708        {
709            let mut state = lock(&self.sender.shared.items);
710            let mut standing = lock(&self.state);
711            if standing.in_flight {
712                standing.queued = Some(notification);
713            } else {
714                standing.in_flight = true;
715                standing.queued = Some(notification);
716                drop(standing);
717                state.flush_latched();
718                // The permanent reservation transfers into the queued entry
719                // (`Entry::standing`) for as long as it sits there, rather
720                // than staying counted in `reserved` on top of the entry's
721                // own `queue.len()` slot -- `take` restores it atomically
722                // with the pop when the entry is eventually drained.
723                state.reserved -= 1;
724                let hold = StandingHold {
725                    // Weak, not Arc: this hold lives inside `Entry` inside
726                    // `state.queue` inside this very `Shared` (PR #20 review
727                    // response) -- a strong reference here would be a
728                    // self-cycle that keeps `Shared` (and its doorbell
729                    // handle) alive forever if this entry is ever left
730                    // permanently undrained.
731                    shared: Arc::downgrade(&self.sender.shared),
732                    state: Arc::clone(&self.state),
733                    resolved: false,
734                };
735                state.queue.push_back(Entry::standing(hold));
736                refresh_doorbell(&self.sender.shared, &state);
737            }
738        }
739        self.sender.shared.arrived.notify_all();
740    }
741}
742
743impl Drop for StandingSlot {
744    /// Lock order: `items`, then `standing` -- matching [`StandingSlot::send`]
745    /// and [`take`] (PR #20 review response), so this can never deadlock
746    /// against either.
747    fn drop(&mut self) {
748        let mut state = lock(&self.sender.shared.items);
749        let mut standing = lock(&self.state);
750        standing.slot_alive = false;
751        if standing.in_flight {
752            // The outstanding hold -- or `take`, inline with the pop -- owns
753            // releasing this reservation once its message drains; finding
754            // the slot already gone, it lets the capacity go free rather
755            // than handing it back here, so this slot has nothing left to
756            // release itself.
757            return;
758        }
759        drop(standing);
760        // As for `Reservation`'s drop: releasing this permanent slot frees
761        // capacity without any item being drained, so a paused producer must
762        // still be prodded on the same zero-to-one transition.
763        state.reserved -= 1;
764        let resumers = freed_resumers(&mut state, true);
765        drop(state);
766        prod(resumers);
767    }
768}
769
770impl std::fmt::Debug for StandingSlot {
771    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
772        f.debug_struct("StandingSlot").finish_non_exhaustive()
773    }
774}
775
776/// The record that a [`StandingSlot::send`] has transferred its permanent
777/// reservation into a queued entry.
778///
779/// The release itself belongs to [`take`], performed inline with the pop under
780/// the same `items` lock (PR #20 review response): popping a standing entry
781/// exposed its queue slot before the reservation was restored under the old
782/// design -- where this type's own `Drop` did the restoring, deferred until
783/// after the queue lock was released -- and a producer woken in that gap could
784/// overcommit capacity. `take` therefore performs the release itself and marks
785/// `resolved`. What is left here is the record of the transfer, which is what
786/// [`StandingSlot::drop`] reads to tell whether it still owns its unit; it is
787/// not a second release path, and this type's `Drop` explains why no discard may
788/// delegate the release back to it.
789struct StandingHold {
790    /// `Weak`, not `Arc` (PR #20 review response): this hold lives inside an
791    /// `Entry` inside `Shared.items.queue` -- inside the very `Shared` a
792    /// strong reference here would point back to, a self-cycle that would
793    /// keep `Shared` (and its doorbell handle) alive forever if this entry
794    /// is ever left permanently undrained (every `Sender`/`Receiver` and
795    /// route dropped before it is). Upgrading fails only when `Shared`
796    /// itself is already being torn down, in which case there is no
797    /// `reserved` accounting left to update.
798    shared: Weak<Shared>,
799    state: Arc<Mutex<StandingState>>,
800    /// Set once [`take`] has performed this hold's release inline with the pop.
801    /// Every hold that leaves the queue leaves it that way, so this is the arm
802    /// `Drop` actually takes; see `Drop` for why the other arm is a tripwire
803    /// rather than a fallback.
804    resolved: bool,
805}
806
807impl Drop for StandingHold {
808    /// A tripwire, not a release path.
809    ///
810    /// Both arms below return, and that is the whole behaviour: a hold is
811    /// reachable only from `state.queue`, the one site that removes an entry
812    /// from that queue ([`take`]) settles the reservation itself and sets
813    /// `resolved`, and the only other way a hold dies is `Shared` being torn
814    /// down, where `upgrade` fails and there is no accounting left to update.
815    ///
816    /// It deliberately does **not** restore the reservation itself, and a future
817    /// discard path must not make it. `take` receives `&mut State`, so its caller
818    /// holds the `items` guard -- and any other way to remove an entry from
819    /// `state.queue` needs that same guard, so a hold discarded on such a path is
820    /// dropped *inside* the lock. Re-taking it here would deadlock, because
821    /// [`lock`] is a plain non-reentrant `Mutex::lock`. That is measured, not
822    /// supposed: with a forced unwind out of `take`, the earlier body that did
823    /// the accounting here hung indefinitely, while the same unwind with this
824    /// `Drop` short-circuited failed immediately. **Any discard path must release
825    /// the reservation under the lock it already holds, exactly as [`take`]
826    /// does.**
827    fn drop(&mut self) {
828        if self.resolved {
829            return;
830        }
831        if self.shared.upgrade().is_none() {
832            // `Shared` is already being torn down; nothing to update.
833            return;
834        }
835        // Reaching here means a hold outlived its entry while the queue was
836        // still alive. Today the only way that can happen is an unwind out of
837        // `take` between the pop and `resolved` being set, and there the original
838        // panic is the real diagnostic -- so it is left to propagate rather than
839        // replaced by an abort from a second panic. Anything else is a new
840        // discard path that has not settled its reservation; see above for the
841        // one way it is allowed to.
842        // **`assert!`, not `debug_assert!`.** A tripwire compiled out of release
843        // builds is not a tripwire: the violation this guards is a future
844        // discard path that forgets to settle its reservation, which would
845        // leave the capacity accounting corrupted and, in release, silently.
846        //
847        // The "no second panic during an unwind" property is preserved by the
848        // predicate rather than by the assertion's flavour: when the thread is
849        // already panicking this passes, so the original panic -- the real
850        // diagnostic -- propagates untouched. It fires only when the hold
851        // outlived its entry *without* an unwind, which is the case that has no
852        // other explanation.
853        assert!(
854            std::thread::panicking(),
855            "a StandingHold outlived its entry with the queue still alive: a \
856             discard path must release the reservation under the `items` lock it \
857             already holds, as `take` does"
858        );
859    }
860}
861
862impl std::fmt::Debug for Reservation {
863    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
864        f.debug_struct("Reservation").finish_non_exhaustive()
865    }
866}
867
868impl Drop for Sender {
869    fn drop(&mut self) {
870        let mut state = lock(&self.shared.items);
871        state.senders -= 1;
872        let last = state.senders == 0;
873        if last {
874            // The end of the stream is something to observe, so the doorbell
875            // reports it too -- otherwise a client waiting on it would wait for a
876            // notification nothing can send.
877            refresh_doorbell(&self.shared, &state);
878        }
879        drop(state);
880        if last {
881            // Wake anyone blocked in `recv`, so a queue that can never be filled
882            // again does not hang its receiver.
883            self.shared.arrived.notify_all();
884        }
885    }
886}
887
888impl std::fmt::Debug for Sender {
889    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
890        f.debug_struct("Sender").finish_non_exhaustive()
891    }
892}
893
894/// The client-side half: the only way to observe notifications.
895pub struct Receiver {
896    shared: Arc<Shared>,
897}
898
899impl Receiver {
900    /// Take the next notification if one is already available.
901    #[must_use]
902    pub fn try_recv(&self) -> Option<Notification> {
903        let (entry, resumers) = {
904            let mut state = lock(&self.shared.items);
905            let taken = take(&mut state);
906            refresh_doorbell(&self.shared, &state);
907            let resumers = freed_resumers(&mut state, taken.is_some());
908            (taken, resumers)
909        };
910        // `entry` (and any `StandingHold` it carries) is dropped here, after
911        // the queue lock above has already been released -- a hold's own
912        // `Drop` re-locks the same mutex to restore its reservation, which
913        // would deadlock if it ran while that lock was still held.
914        prod(resumers);
915        entry.map(Entry::into_notification)
916    }
917
918    /// Block until a notification is available, or every sender is gone.
919    ///
920    /// Returns `None` only when nothing is queued, nothing is latched, *and* no
921    /// sender remains, so a client loop terminates on teardown instead of
922    /// hanging -- and never before it has been told about a loss.
923    #[must_use]
924    pub fn recv(&self) -> Option<Notification> {
925        let mut state = lock(&self.shared.items);
926        loop {
927            if let Some(entry) = take(&mut state) {
928                refresh_doorbell(&self.shared, &state);
929                let resumers = freed_resumers(&mut state, true);
930                drop(state);
931                prod(resumers);
932                // As in `try_recv`: `entry` is dropped only after the lock
933                // above is released.
934                return Some(entry.into_notification());
935            }
936            if state.senders == 0 {
937                return None;
938            }
939            state = self
940                .shared
941                .arrived
942                .wait(state)
943                .unwrap_or_else(|poison| poison.into_inner());
944        }
945    }
946
947    /// Block for at most `timeout`.
948    ///
949    /// Returns `None` on timeout as well as on teardown; a caller that must tell
950    /// them apart can check [`Receiver::is_disconnected`].
951    #[must_use]
952    pub fn recv_timeout(&self, timeout: Duration) -> Option<Notification> {
953        let deadline = Instant::now() + timeout;
954        let mut state = lock(&self.shared.items);
955        loop {
956            if let Some(entry) = take(&mut state) {
957                refresh_doorbell(&self.shared, &state);
958                let resumers = freed_resumers(&mut state, true);
959                drop(state);
960                prod(resumers);
961                // As in `try_recv`: `entry` is dropped only after the lock
962                // above is released.
963                return Some(entry.into_notification());
964            }
965            if state.senders == 0 {
966                return None;
967            }
968            let remaining = deadline.checked_duration_since(Instant::now())?;
969            let (next, _) = self
970                .shared
971                .arrived
972                .wait_timeout(state, remaining)
973                .unwrap_or_else(|poison| poison.into_inner());
974            state = next;
975        }
976    }
977
978    /// A manual-reset event that is signalled whenever this receiver has
979    /// something to take.
980    ///
981    /// This is what lets a client integrate with its own thread pool instead of
982    /// dedicating a thread to a blocking [`Receiver::recv`]: wait on the handle,
983    /// then drain with [`Receiver::try_recv`] until it yields `None`.
984    ///
985    /// The event is created on the first call, so a client that never asks for it
986    /// pays for no kernel object. It stays signalled while anything is
987    /// outstanding, including once the stream has ended -- so a waiter learns
988    /// about disconnection rather than waiting for a notification that can never
989    /// arrive.
990    ///
991    /// The borrow is deliberate: the event belongs to this queue and must not be
992    /// closed by a caller. Use [`Receiver::doorbell_shared`] or
993    /// [`Receiver::doorbell_owned`] where ownership is required, such as arming a
994    /// `ThreadpoolWait`.
995    ///
996    /// # Errors
997    ///
998    /// Returns the error from `CreateEventW` on the first call.
999    pub fn doorbell(&self) -> io::Result<BorrowedHandle<'_>> {
1000        self.ensure_doorbell()?;
1001        Ok(self
1002            .shared
1003            .doorbell
1004            .get()
1005            .expect("the doorbell was just created")
1006            .as_handle())
1007    }
1008
1009    /// A duplicate of [`Receiver::doorbell`] that the caller owns.
1010    ///
1011    /// The duplicate refers to the same event, so signalling reaches both; the
1012    /// caller closes its own copy whenever it likes. This is the form a
1013    /// `ThreadpoolWait` needs, since arming one takes ownership of its target.
1014    ///
1015    /// # Errors
1016    ///
1017    /// Returns the error from `CreateEventW` or `DuplicateHandle`.
1018    pub fn doorbell_owned(&self) -> io::Result<OwnedHandle> {
1019        duplicate_handle(
1020            self.doorbell()?,
1021            DuplicateAccess::Same,
1022            Inheritance::NotInheritable,
1023        )
1024    }
1025
1026    /// The doorbell as a shared handle: the queue's own event, held by one more
1027    /// owner.
1028    ///
1029    /// This is [`Receiver::doorbell`] with the ownership the borrow lacks, and no
1030    /// new kernel handle: the result is a clone of the handle the queue holds, so
1031    /// its raw handle is the borrow's, and the event closes when the last holder
1032    /// of it, the queue's own included, drops. A `ThreadpoolWait` can take it
1033    /// through `WaitableHandle::assume_waitable_shared`, which is what arming one
1034    /// needs without a duplicate per waiter.
1035    ///
1036    /// The queue keeps the event in agreement with its contents only for as long as
1037    /// the queue exists. Holding this does not keep the queue alive, and nothing
1038    /// signals or resets the event once the queue is gone, so a holder that outlives
1039    /// every sender and the receiver is waiting on an event nothing will change. Use
1040    /// [`Receiver::doorbell_owned`] for a handle that is the caller's own and can be
1041    /// closed whenever the caller likes.
1042    ///
1043    /// # Errors
1044    ///
1045    /// Returns the error from `CreateEventW` on the first call to any doorbell
1046    /// method.
1047    pub fn doorbell_shared(&self) -> io::Result<SharedHandle> {
1048        self.ensure_doorbell()?;
1049        Ok(self
1050            .shared
1051            .doorbell
1052            .get()
1053            .expect("the doorbell was just created")
1054            .clone())
1055    }
1056
1057    /// Create the doorbell if it does not exist yet.
1058    fn ensure_doorbell(&self) -> io::Result<()> {
1059        if self.shared.doorbell.get().is_some() {
1060            return Ok(());
1061        }
1062        // Created under the queue lock so its initial state cannot disagree with
1063        // the queue it reports on: a client that asks for a doorbell after
1064        // notifications have already arrived must find it signalled.
1065        let state = lock(&self.shared.items);
1066        if self.shared.doorbell.get().is_none() {
1067            let event = create_event(state.pending())?;
1068            let _ = self.shared.doorbell.set(SharedHandle::new(event));
1069        }
1070        Ok(())
1071    }
1072
1073    /// Whether every sender has been dropped, so nothing further can arrive.
1074    #[must_use]
1075    pub fn is_disconnected(&self) -> bool {
1076        lock(&self.shared.items).senders == 0
1077    }
1078
1079    /// How many notifications are queued right now.
1080    ///
1081    /// Excludes latched losses, which are not queued -- that is the point of the
1082    /// latch. [`Receiver::latched`] counts those.
1083    ///
1084    /// **This is not "how much is there to take".** A drained queue with a loss
1085    /// still owed reports `0` here while [`Receiver::recv`] would still yield a
1086    /// synthesised `Desync { QueueFull }`. Use [`Receiver::has_pending`] to ask
1087    /// whether anything is collectable.
1088    #[must_use]
1089    pub fn len(&self) -> usize {
1090        lock(&self.shared.items).queue.len()
1091    }
1092
1093    /// Whether nothing is queued right now.
1094    ///
1095    /// **Not the same as "nothing to collect"**, and the difference is a live
1096    /// hazard rather than a nicety: this excludes owed latched losses and the
1097    /// end of the stream, both of which [`Receiver::recv`] still reports. A
1098    /// drain loop written as `while !receiver.is_empty()` therefore exits with a
1099    /// `Desync { QueueFull }` still owed -- and, because the doorbell stays
1100    /// signalled while a loss is owed (D-41), a client that waits on the
1101    /// doorbell and then tests this spins without ever collecting the report it
1102    /// was woken for. Use [`Receiver::has_pending`], which is the predicate the
1103    /// doorbell is actually signalled on.
1104    #[must_use]
1105    pub fn is_empty(&self) -> bool {
1106        self.len() == 0
1107    }
1108
1109    /// Whether a [`Receiver::recv`] would produce something rather than block.
1110    ///
1111    /// This is *the* predicate the doorbell is signalled on (D-41), so it is the
1112    /// one to test after a doorbell wait. "Something to take" is deliberately
1113    /// three things, not one:
1114    ///
1115    /// - a queued notification ([`Receiver::len`]);
1116    /// - an **owed latched loss** ([`Receiver::latched`]) -- not in the queue,
1117    ///   but still collectable, and synthesised on demand (D-39);
1118    /// - the **end of the stream** ([`Receiver::is_disconnected`]) -- a client
1119    ///   must learn the stream ended rather than wait for a notification
1120    ///   nothing can send.
1121    ///
1122    /// Prefer this over [`Receiver::is_empty`] anywhere the question is "is
1123    /// there anything for me?" rather than "how deep is the queue?".
1124    #[must_use]
1125    pub fn has_pending(&self) -> bool {
1126        lock(&self.shared.items).pending()
1127    }
1128
1129    /// How many subscriptions are owed a `Desync { QueueFull }`.
1130    #[must_use]
1131    pub fn latched(&self) -> usize {
1132        lock(&self.shared.items).latched.len()
1133    }
1134
1135    /// The queue's bound.
1136    #[must_use]
1137    pub fn capacity(&self) -> usize {
1138        lock(&self.shared.items).capacity
1139    }
1140}
1141
1142/// Take the next item: a queued notification, or -- once the queue is drained --
1143/// a synthesised report of a loss that never fitted.
1144///
1145/// The queue comes first because the latch records a loss that happened *after*
1146/// everything still queued: the queue was full at the time, so surfacing the
1147/// desync early would claim the hole is older than it is and break the ordering
1148/// D-12 promises.
1149/// Take the next item: a queued entry, or -- once the queue is drained -- a
1150/// synthesised report of a loss that never fitted.
1151///
1152/// The queue comes first because the latch records a loss that happened *after*
1153/// everything still queued: the queue was full at the time, so surfacing the
1154/// desync early would claim the hole is older than it is and break the ordering
1155/// D-12 promises.
1156///
1157/// A standing entry's reservation is restored right here, inline with the pop,
1158/// under the same `items` lock `state` already holds (PR #20 review response):
1159/// popping the entry first and restoring the reservation later (in the
1160/// `StandingHold`'s own deferred `Drop`) left a window where `queue.len()` had
1161/// already dropped but `reserved` had not yet gone back up, so `free()`
1162/// over-reported room by one and a producer woken in that gap could overcommit
1163/// capacity. The returned `Entry` still carries its (now inert) `StandingHold`
1164/// so a caller may defer *dropping* it until after the queue lock is released
1165/// -- the hold's own `Drop` is a no-op here, since `resolved` is already set --
1166/// but the reservation itself is never left outstanding past this call. Setting
1167/// `resolved` is not bookkeeping that can be skipped: it is what keeps the hold
1168/// off the tripwire in `StandingHold::drop`, which any future path that removes
1169/// an entry from the queue must satisfy the same way -- by releasing inline,
1170/// here, under the lock it already holds.
1171fn take(state: &mut State) -> Option<Entry> {
1172    if let Some(mut entry) = state.queue.pop_front() {
1173        if matches!(entry.notification, EntryNotification::Standing) {
1174            let hold = entry
1175                ._standing_hold
1176                .as_mut()
1177                .expect("a Standing entry always carries a StandingHold");
1178            let notification = {
1179                let mut standing = lock(&hold.state);
1180                standing.in_flight = false;
1181                let notification = standing.queued.take().expect(
1182                    "a Standing entry's queued value is set for as long as it is undrained",
1183                );
1184                if standing.slot_alive {
1185                    state.reserved += 1;
1186                }
1187                notification
1188            };
1189            hold.resolved = true;
1190            entry.notification = EntryNotification::Plain(notification);
1191        }
1192        return Some(entry);
1193    }
1194    state.latched.pop_front().map(|watch| {
1195        Entry::plain(Notification::Desync {
1196            watch,
1197            cause: DesyncCause::QueueFull,
1198        })
1199    })
1200}
1201
1202impl std::fmt::Debug for Receiver {
1203    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
1204        let state = lock(&self.shared.items);
1205        f.debug_struct("Receiver")
1206            .field("queued", &state.queue.len())
1207            .field("capacity", &state.capacity)
1208            .field("latched", &state.latched.len())
1209            .field("disconnected", &(state.senders == 0))
1210            .finish_non_exhaustive()
1211    }
1212}
1213
1214/// Create a connected sender/receiver pair with the default bound.
1215///
1216/// A client reaches a queue through [`Monitor::session`](crate::monitor::Monitor::session),
1217/// which chooses the bound; this exists for tests that need a queue without one.
1218#[cfg(test)]
1219pub(crate) fn channel() -> (Sender, Receiver) {
1220    channel_with_bound(DEFAULT_BOUND)
1221}
1222
1223/// Create a connected sender/receiver pair holding at most `bound` notifications.
1224///
1225/// The bound is a [`NonZeroUsize`] rather than a checked `usize` because a zero
1226/// bound is not a runtime condition to report -- it is a queue that could not
1227/// carry even the desync announcing its own saturation, making the crate's
1228/// never-silently-lose guarantee vacuous. Making it unrepresentable rejects it at
1229/// construction, where D-11 asks for it, and leaves no error path to handle.
1230///
1231/// This is also the consumer test seam (D-81/D-82), reachable by a downstream
1232/// consumer only under the `test-util` feature: a consumer that wants to test its
1233/// own notification-handling code constructs a pair here, pushes synthetic
1234/// notifications through the [`Sender`] with [`Sender::send`], and drains the
1235/// [`Receiver`] exactly as it would one from
1236/// [`Monitor::session`](crate::Monitor::session) -- with no real filesystem or
1237/// thread pool. See the crate-level "Testing your consumer code".
1238pub fn channel_with_bound(bound: NonZeroUsize) -> (Sender, Receiver) {
1239    let shared = Arc::new(Shared {
1240        items: Mutex::new(State {
1241            queue: VecDeque::new(),
1242            capacity: bound.get(),
1243            reserved: 0,
1244            latched: VecDeque::new(),
1245            senders: 1,
1246            resumers: Vec::new(),
1247        }),
1248        arrived: Condvar::new(),
1249        doorbell: OnceLock::new(),
1250    });
1251    (
1252        Sender {
1253            shared: Arc::clone(&shared),
1254        },
1255        Receiver { shared },
1256    )
1257}
1258
1259/// The producers to prod, if taking an item has just made room.
1260///
1261/// Returns them rather than calling them, because a producer's `resume` must not
1262/// run under the queue lock: the obvious implementation re-arms a read, and a
1263/// failed arm reports itself by *sending*, which would take this very lock.
1264///
1265/// Dead producers are pruned here, which is the only place the list is walked.
1266fn freed_resumers(state: &mut State, took_one: bool) -> Vec<Arc<dyn Resume>> {
1267    // Only on the transition into having room: prodding on every take would put
1268    // a wake behind every single notification a client drains. The edge is
1269    // expressed in `best_effort_room` -- the same quantity `Sender::has_room`
1270    // tests -- so the prod fires exactly when a parked producer's re-check will
1271    // now succeed. Every step that frees capacity (taking a queued item,
1272    // synthesising a latched one, releasing a reservation) raises it by exactly
1273    // one, so `== 1` catches the crossing and cannot be stepped over.
1274    if !took_one || state.resumers.is_empty() || state.best_effort_room() != 1 {
1275        return Vec::new();
1276    }
1277    let mut live = Vec::with_capacity(state.resumers.len());
1278    let mut awake = Vec::new();
1279    for weak in state.resumers.drain(..) {
1280        if let Some(producer) = weak.upgrade() {
1281            live.push(Arc::downgrade(&producer));
1282            awake.push(producer);
1283        }
1284    }
1285    state.resumers = live;
1286    awake
1287}
1288
1289/// Tell each producer there may be room now.
1290fn prod(resumers: Vec<Arc<dyn Resume>>) {
1291    for producer in resumers {
1292        producer.resume();
1293    }
1294}
1295
1296/// Lock, recovering the guard if a previous holder panicked.
1297///
1298/// A poisoned lock here means some thread panicked while holding it; the queue is
1299/// a plain `VecDeque` plus a count, both of which are left structurally intact by
1300/// every path, so refusing to proceed would strand the receiver rather than
1301/// protect anything.
1302fn lock<T>(mutex: &Mutex<T>) -> std::sync::MutexGuard<'_, T> {
1303    mutex.lock().unwrap_or_else(|poison| poison.into_inner())
1304}
1305
1306/// Bring the doorbell into agreement with the state.
1307///
1308/// Every mutation ends here, under the same lock a receiver holds while deciding
1309/// there is nothing to take -- which is what makes a lost wakeup impossible
1310/// rather than merely unlikely: there is no window between the two decisions for
1311/// one to fall into.
1312fn refresh_doorbell(shared: &Shared, state: &State) {
1313    let Some(doorbell) = shared.doorbell.get() else {
1314        // Never asked for, so there is nothing to keep in agreement.
1315        return;
1316    };
1317    let raw = doorbell.as_raw_handle();
1318    // SAFETY: `raw` is a live manual-reset event owned by this queue for as long
1319    // as the queue exists, and neither call has any other precondition.
1320    unsafe {
1321        if state.pending() {
1322            SetEvent(raw);
1323        } else {
1324            ResetEvent(raw);
1325        }
1326    }
1327}
1328
1329/// Create a manual-reset event in the given initial state.
1330fn create_event(signalled: bool) -> io::Result<OwnedHandle> {
1331    // SAFETY: creates an unnamed event with default security attributes; both
1332    // pointer arguments are null by design.
1333    let raw = unsafe {
1334        CreateEventW(
1335            ptr::null(),
1336            TRUE,
1337            if signalled { TRUE } else { FALSE },
1338            ptr::null(),
1339        )
1340    };
1341    if raw.is_null() {
1342        return Err(io::Error::last_os_error());
1343    }
1344    // SAFETY: the call returned a fresh, exclusively owned event handle.
1345    Ok(unsafe { OwnedHandle::from_raw_handle(raw) })
1346}
1347
1348#[cfg(test)]
1349mod tests;