windows_file_watcher/queue.rs
1// Copyright (c) 2026 Mike Grier
2//! The crate-owned notification queue: where a watcher puts what it decodes, and
3//! where a client takes it from.
4//!
5//! The queue is the boundary that keeps client behaviour off this crate's cadence
6//! path. Delivery is an enqueue *the crate performs*; the client only ever
7//! receives. There is no sink trait and no client-supplied closure, so nothing a
8//! client does -- blocking, panicking, being slow -- can stall or unwind a
9//! completion callback (D-2/D-11).
10//!
11//! Note this is a statement about the **call graph**, not about threads. Which
12//! thread a client drains on is entirely its own business, and draining from its
13//! own thread-pool callback is an expected integration; that is the client's pool
14//! object and the client's cadence. The M3.4 doorbell (D-25) exists to make
15//! precisely that integration possible without dedicating a thread.
16//!
17//! # Reserved is guaranteed, unreserved is best-effort
18//!
19//! The queue is bounded, so it can fill, and the two kinds of thing that travel
20//! it cannot survive that the same way. A lost change notification is recoverable
21//! -- re-scanning is exactly what [`DesyncCause`] exists to ask for -- while a
22//! lost completion or fault report is a liveness bug, because the client waits
23//! forever for something that already happened.
24//!
25//! Rather than sort that out per message type at the enqueue, reliability is a
26//! property of **reserved capacity** (D-33). A control message takes its slot
27//! through [`Sender::reserve`] *before* whatever produces it is allowed to
28//! proceed, so [`Reservation::send`] cannot fail -- the room is already the
29//! sender's. Change notifications reserve nothing and go through
30//! [`Sender::send`], which is allowed to fail. One line covers both: *reserved is
31//! guaranteed, unreserved is best-effort.*
32//!
33//! # Reporting a full queue without needing room to do it
34//!
35//! A desync is an ordinary notification and rides the queue in order like any
36//! other (D-12/D-26), which is what lets a client say "everything before this is
37//! accounted for". That breaks down in exactly one place: reporting that the
38//! queue is full cannot itself require a slot in the full queue, and reserving
39//! one only defers the problem to the second overflow.
40//!
41//! So a failed enqueue latches the affected [`WatchId`] in a set held *outside*
42//! the bounded queue, where it coalesces -- lossless, because a desync is
43//! idempotent and the answer is always a re-scan. The latch is drained back into
44//! the queue at the next successful enqueue, which is also the position the loss
45//! actually occupies: the queue was full when the loss happened, so everything
46//! still queued precedes it and everything enqueued afterwards follows it.
47//! A receiver that reaches an empty queue synthesises any remainder directly, so
48//! a latched desync is delivered even if nothing further is ever sent.
49//!
50//! # The doorbell
51//!
52//! A queue drainable only by a blocking [`Receiver::recv`] would force a client
53//! that already owns a thread pool to dedicate a thread to it -- contradicting
54//! this crate's premise that nobody should have to own threads. So the receiver
55//! hands out a manual-reset event ([`Receiver::doorbell`]) that a client can wait
56//! on however it likes, including from its own `ThreadpoolWait`.
57//!
58//! It is created **lazily**, so a client that only ever calls `recv` allocates no
59//! kernel object at all.
60//!
61//! *Why not a `Doorbell` trait the client implements?* Because that would be a
62//! client callback on this crate's cadence path, which is the one thing D-2
63//! forbids, and it would have made `Monitor`, `Session`, and `Sender` all generic
64//! over it. The composition argument for a trait does not survive contact with
65//! the platform either: on Windows a HANDLE **is** the universal waitable
66//! currency -- `WaitForSingleObject`, `WaitForMultipleObjects`,
67//! `MsgWaitForMultipleObjects`, `ThreadpoolWait` and alertable waits all take one
68//! -- so an event is the native composition point rather than a lowest common
69//! denominator. The single case it does not reach, an async `Waker`, is a short
70//! bridge the client writes on its own pool, which is where that code belongs.
71//! Owning the doorbell also makes the reset discipline an internal invariant
72//! rather than a client obligation (D-25).
73//!
74//! That invariant is one line: **the event is signalled exactly when the receiver
75//! has something to observe**, which is a queued notification, an owed loss
76//! report, or the end of the stream. It is re-established under the queue lock at
77//! the end of every mutation -- the same lock a receiver holds while deciding
78//! there is nothing to take -- so a wakeup cannot be lost in the gap between
79//! those two decisions, because there is no gap.
80//!
81//! This is the crate-owned notification queue. A client holds the [`Receiver`]
82//! and drains it; nothing in this module is reachable from client code except
83//! through that.
84
85use std::collections::VecDeque;
86use std::io;
87use std::num::NonZeroUsize;
88use std::os::windows::io::{AsHandle, AsRawHandle, BorrowedHandle, FromRawHandle, OwnedHandle};
89use std::ptr;
90use std::sync::{Arc, Condvar, Mutex, OnceLock, Weak};
91use std::time::{Duration, Instant};
92
93use win_handles_sys::{DuplicateAccess, Inheritance, duplicate_handle};
94use win_shared_os_owned_handle::SharedHandle;
95use windows_sys::Win32::Foundation::{FALSE, TRUE};
96use windows_sys::Win32::System::Threading::{CreateEventW, ResetEvent, SetEvent};
97
98use crate::directory::{FaultDetail, VolumeIdentity};
99use crate::notify::{Change, DesyncCause};
100use crate::retry::{FaultOperation, WatchMode};
101
102/// The bound used when a caller does not choose one.
103///
104/// Counts **notifications**, not changes: one decoded completion is one batch
105/// (D-10) and a batch can carry hundreds of records, so this is a far deeper
106/// backlog than the number suggests. It is chosen to be ample for a client that
107/// drains promptly and still bounded for one that stops.
108pub const DEFAULT_BOUND: NonZeroUsize = match NonZeroUsize::new(1024) {
109 Some(bound) => bound,
110 None => unreachable!(),
111};
112
113/// Identifies the subscription a notification belongs to.
114///
115/// `Copy`, so a client can retain it to route or aggregate without holding the
116/// subscription's lifecycle object (D-5). M3.4 issues these from the monitor;
117/// until then a watcher is constructed with one.
118#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)]
119pub struct WatchId(u64);
120
121impl WatchId {
122 /// Build an identifier from a raw value.
123 ///
124 /// In production the monitor issues `WatchId`s. This constructor lets a
125 /// downstream consumer mint one to tag a synthetic [`Notification`] when
126 /// testing its own handler against a `Receiver` it drives (D-81); any value
127 /// is a valid identifier, so the pairing between an id and its notifications
128 /// is the consumer's to choose.
129 #[must_use]
130 pub fn from_raw(value: u64) -> Self {
131 Self(value)
132 }
133
134 /// The raw value, for a client that wants to key a map on it.
135 #[must_use]
136 pub fn get(self) -> u64 {
137 self.0
138 }
139}
140
141/// How a request turned out.
142///
143/// Registration is asynchronous (D-2), so the outcome cannot be the return value
144/// of the call that made the request; it arrives here instead.
145#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
146pub enum Outcome {
147 /// The subscription is registered and the monitor is watching for it.
148 Subscribed,
149 /// The subscription is registered, but the target could not be opened yet.
150 ///
151 /// Retryable in D-22's sense -- a path that does not exist yet, a network
152 /// share that is briefly unreachable -- so the monitor keeps the
153 /// subscription and will establish it when it can. This is not a failure and
154 /// carries no terminal meaning (D-14).
155 Establishing,
156 /// The target can never be watched, so nothing was registered.
157 ///
158 /// Only D-22's permanent pair reaches here, and both mean the caller named
159 /// something that is not a watchable directory rather than that anything went
160 /// wrong in the environment. Retrying would spin forever against input that
161 /// will never become valid.
162 Failed {
163 /// The classification and raw code behind the permanent failure (D-79).
164 detail: FaultDetail,
165 },
166 /// The subscription has ended and its watcher is released.
167 ///
168 /// Its position in the stream is the guarantee: everything before it belongs
169 /// to the live watch, and nothing after it does (D-30).
170 Cancelled,
171}
172
173/// One item a client receives, always tagged with the subscription it belongs to.
174///
175/// Changes and desyncs ride the same queue so their order relative to one another
176/// is well defined within a subscription (D-12): a client that sees a `Desync`
177/// knows every change enqueued before it, and none after, is accounted for.
178#[derive(Clone, Debug, PartialEq, Eq)]
179pub enum Notification {
180 /// The changes one completion carried, in the order the kernel reported them.
181 Batch {
182 /// The subscription these changes belong to.
183 watch: WatchId,
184 /// The changes, in kernel order.
185 changes: Vec<Change>,
186 },
187 /// Changes were lost; the client should re-scan (D-12).
188 Desync {
189 /// The subscription affected.
190 watch: WatchId,
191 /// How the gap arose. Advisory for the four recoverable causes -- the
192 /// response is a re-scan either way -- but
193 /// [`DesyncCause::Stopped`] is terminal and must be matched on.
194 cause: DesyncCause,
195 },
196 /// A request the client made has been serviced (D-30).
197 ///
198 /// Carried on this queue rather than a side channel so that its ordering
199 /// against data is **structural** rather than temporal.
200 Completion {
201 /// The subscription the request concerned.
202 watch: WatchId,
203 /// What happened.
204 outcome: Outcome,
205 },
206 /// The watch stopped delivering while the monitor works to re-establish it
207 /// (D-14/D-15/D-31). Opt-in via `WatchOptions::report_liveness` (D-13).
208 Suspended {
209 /// The subscription affected.
210 watch: WatchId,
211 },
212 /// The watch is delivering again after a [`Notification::Suspended`]. A
213 /// [`Notification::Desync { Reestablished }`](DesyncCause::Reestablished)
214 /// always precedes or accompanies this, since the gap it bridges may have
215 /// hidden changes. Opt-in (D-13).
216 Resumed {
217 /// The subscription affected.
218 watch: WatchId,
219 },
220 /// Which tier is actually watching this subscription's target, reported
221 /// once at first establishment and again after every re-establishment:
222 /// [`WatchMode::Detailed`] or the [`WatchMode::Coarse`] fallback (D-17).
223 /// Opt-in (D-13).
224 Established {
225 /// The subscription affected.
226 watch: WatchId,
227 /// The tier now watching.
228 mode: WatchMode,
229 },
230 /// The monitor is asking this **interactive** subscription (D-27) how long
231 /// to wait before the next attempt. Answer through
232 /// [`crate::session::Session::answer`]; an explicit `answer(watch, None)`
233 /// declines, which is counted at the operation's default delay -- never
234 /// answering at all leaves the question outstanding indefinitely. This is
235 /// the one message a fault protocol cannot afford to lose, so it rides a
236 /// standing reservation (`Sender::reserve_standing`) taken once at
237 /// registration rather than the best-effort path -- sound because a watcher
238 /// cannot fault twice concurrently (D-28), so at most one question per
239 /// subscription is ever outstanding.
240 RetryQuestion {
241 /// The subscription being asked.
242 watch: WatchId,
243 /// Which operation faulted.
244 operation: FaultOperation,
245 /// The classification and raw code behind the failure (D-79).
246 detail: FaultDetail,
247 },
248 /// A path-based reopen landed on a different volume than this
249 /// subscription started on, and it opted into
250 /// [`crate::VolumeChangePolicy::Confirm`] (D-78/M12). Answer through
251 /// [`crate::session::Session::answer_volume_change`];
252 /// [`crate::VolumeChangeDecision::Stop`] removes just this subscription,
253 /// [`crate::VolumeChangeDecision::Continue`] keeps it running against the
254 /// new volume. Rides the same standing reservation `RetryQuestion` does
255 /// (never outstanding at the same time for one subscription, D-28).
256 VolumeChanged {
257 /// The subscription being asked.
258 watch: WatchId,
259 /// The volume this subscription was previously confirmed on.
260 previous: VolumeIdentity,
261 /// The volume the reopen just landed on.
262 current: VolumeIdentity,
263 },
264}
265
266impl Notification {
267 /// The subscription this notification belongs to.
268 #[must_use]
269 pub fn watch(&self) -> WatchId {
270 match self {
271 Notification::Batch { watch, .. }
272 | Notification::Desync { watch, .. }
273 | Notification::Completion { watch, .. }
274 | Notification::Suspended { watch }
275 | Notification::Resumed { watch }
276 | Notification::Established { watch, .. }
277 | Notification::RetryQuestion { watch, .. }
278 | Notification::VolumeChanged { watch, .. } => *watch,
279 }
280 }
281}
282
283/// The shared queue storage.
284struct Shared {
285 items: Mutex<State>,
286 arrived: Condvar,
287 /// The client-facing wake, created on first request and never replaced.
288 ///
289 /// Outside the lock so a borrow of it can outlive the guard, but only ever
290 /// signalled or reset *under* the lock -- see [`refresh_doorbell`].
291 doorbell: OnceLock<SharedHandle>,
292}
293
294/// A crate-owned producer that stopped because the queue was full, and must be
295/// prodded when there is room again.
296///
297/// This is **not** the client callback D-2 forbids and D-25 rejected: every
298/// implementor is this crate's own code, and the contract is one line -- return
299/// promptly, touch nothing of the queue's. Observation is the only tier that
300/// needs it, because it is the only one that reserves nothing (D-33).
301pub(crate) trait Resume: Send + Sync {
302 /// There may be room now. Called with no lock of the queue's held.
303 fn resume(&self);
304}
305
306/// One queued notification, optionally carrying a [`StandingSlot`]'s permanent
307/// reservation for as long as it sits in the queue.
308///
309/// A `StandingSlot` send transfers its one carved-out unit of `reserved`
310/// capacity into the queue for as long as the message sits there (so `free()`
311/// is never double-charged for it, once as `reserved` and again as an
312/// occupied queue slot). [`take`] hands that unit back, inline with the pop
313/// that removes this entry; the [`StandingHold`] is the record that the
314/// transfer happened, which is what lets [`StandingSlot::drop`] tell whether it
315/// still owns that unit itself (no hold outstanding) or has already
316/// transferred it to a queued entry that has not been drained yet.
317struct Entry {
318 notification: EntryNotification,
319 _standing_hold: Option<StandingHold>,
320}
321
322/// A plain entry's notification is fixed at push time; a standing entry's
323/// lives in its paired [`StandingHold`]'s [`StandingState::queued`] instead,
324/// so a second [`StandingSlot::send`] while this entry is still undrained can
325/// coalesce into it in place (PR #20 review response) rather than pushing a
326/// second entry and double-spending the slot's single unit of `reserved`.
327enum EntryNotification {
328 Plain(Notification),
329 Standing,
330}
331
332impl Entry {
333 fn plain(notification: Notification) -> Self {
334 Self {
335 notification: EntryNotification::Plain(notification),
336 _standing_hold: None,
337 }
338 }
339
340 fn standing(hold: StandingHold) -> Self {
341 Self {
342 notification: EntryNotification::Standing,
343 _standing_hold: Some(hold),
344 }
345 }
346
347 /// The notification this entry carries, reading a standing entry's
348 /// current (possibly coalesced) value from its `StandingHold` before the
349 /// entry itself is discarded.
350 fn into_notification(self) -> Notification {
351 match self.notification {
352 EntryNotification::Plain(notification) => notification,
353 EntryNotification::Standing => {
354 let hold = self
355 ._standing_hold
356 .as_ref()
357 .expect("a Standing entry always carries a StandingHold");
358 lock(&hold.state)
359 .queued
360 .clone()
361 .expect("a Standing entry's queued value is set for as long as it is undrained")
362 }
363 }
364 }
365}
366
367struct State {
368 queue: VecDeque<Entry>,
369 /// The bound. Never zero, which is what makes the delivery guarantee
370 /// non-vacuous: a queue with no room could never carry the desync that
371 /// reports its own saturation.
372 capacity: usize,
373 /// Slots taken by an outstanding [`Reservation`] but not yet filled. Held
374 /// away from the best-effort path, which is the whole mechanism: a reserved
375 /// send cannot fail because nothing else can take its room.
376 reserved: usize,
377 /// Subscriptions with a loss that could not be enqueued, in the order they
378 /// were first latched.
379 ///
380 /// Deliberately outside the bounded queue, and deliberately coalesced: a
381 /// second loss for a subscription that already owes a desync adds nothing,
382 /// because the client's response to one is the same as its response to ten.
383 latched: VecDeque<WatchId>,
384 /// Set when every sender is gone, so a blocked receiver can stop waiting
385 /// rather than hang forever on a queue nothing can fill.
386 senders: usize,
387 /// Producers to prod when a saturated queue frees a slot.
388 ///
389 /// Weak, so a producer that has gone away is simply forgotten rather than
390 /// kept alive by the queue it fed.
391 resumers: Vec<Weak<dyn Resume>>,
392}
393
394impl State {
395 /// Slots available to the best-effort path: neither occupied nor reserved.
396 fn free(&self) -> usize {
397 self.capacity - self.queue.len() - self.reserved
398 }
399
400 /// Slots a best-effort notification could actually take *right now*.
401 ///
402 /// [`Sender::send`] flushes every owed latch into the queue before it
403 /// considers the caller's own notification, so a free slot with a latch
404 /// outstanding already belongs to that flush. This is therefore
405 /// `free()` minus what is owed, and it is the **single** definition both
406 /// [`Sender::has_room`] (`> 0`) and [`freed_resumers`]' wake edge (`== 1`)
407 /// are expressed in terms of.
408 ///
409 /// Keeping them derived from one quantity is deliberate: they were
410 /// previously two independent expressions, `free() > latched.len()` and
411 /// `free() == 1`, which disagreed the moment anything was latched -- the
412 /// prod fired one slot too early, the re-check still said "no room", and no
413 /// later drain re-fired it, wedging the watcher permanently
414 /// (PR #42 review). A predicate and the edge that wakes it cannot be
415 /// allowed to drift apart.
416 fn best_effort_room(&self) -> usize {
417 self.free().saturating_sub(self.latched.len())
418 }
419
420 /// Whether a receiver has anything to observe.
421 ///
422 /// Disconnection counts: a client waiting on the doorbell must learn that the
423 /// stream has ended, or it would wait for a notification nothing can send.
424 fn pending(&self) -> bool {
425 !self.queue.is_empty() || !self.latched.is_empty() || self.senders == 0
426 }
427
428 /// Move latched losses back into the queue, as many as there is room for.
429 ///
430 /// Called immediately before an enqueue, which is what puts each desync in
431 /// the position the loss actually occupies -- see the module docs.
432 fn flush_latched(&mut self) {
433 while self.free() > 0 {
434 let Some(watch) = self.latched.pop_front() else {
435 break;
436 };
437 self.queue.push_back(Entry::plain(Notification::Desync {
438 watch,
439 cause: DesyncCause::QueueFull,
440 }));
441 }
442 }
443
444 /// Record that a subscription lost something, coalescing with any loss it is
445 /// already owed.
446 fn latch(&mut self, watch: WatchId) {
447 if !self.latched.contains(&watch) {
448 self.latched.push_back(watch);
449 }
450 }
451}
452
453/// What became of a best-effort notification.
454#[derive(Clone, Copy, Debug, PartialEq, Eq)]
455#[must_use = "a latched notification was lost, which M3.7 responds to by not re-arming"]
456pub enum Delivery {
457 /// Enqueued. The client will receive it.
458 Queued,
459 /// The queue was full, so it was dropped and a `Desync { QueueFull }` is now
460 /// owed to its subscription.
461 Latched,
462}
463
464/// The crate-side half: enqueues, never blocks.
465///
466/// Cloneable and `Send + Sync` because several watchers -- whose completions run
467/// on different pool threads -- feed one client queue (D-11).
468pub struct Sender {
469 shared: Arc<Shared>,
470}
471
472impl Clone for Sender {
473 fn clone(&self) -> Self {
474 let mut state = lock(&self.shared.items);
475 state.senders += 1;
476 drop(state);
477 Self {
478 shared: Arc::clone(&self.shared),
479 }
480 }
481}
482
483impl Sender {
484 /// Enqueue one notification, best-effort.
485 ///
486 /// Runs on the cadence path, so it never blocks. It reserves nothing, so it
487 /// *can* fail: a full queue drops the notification and latches a
488 /// `Desync { QueueFull }` against its subscription (D-29/D-33). Use
489 /// [`Sender::reserve`] for anything whose loss a client cannot recover from.
490 ///
491 /// A downstream consumer may call this on a `Sender` from
492 /// [`channel_with_bound`] to feed its own handler synthetic notifications in
493 /// a test (behind the `test-util` feature); the returned [`Delivery`] reports
494 /// whether the queue accepted the notification or dropped-and-latched it,
495 /// exactly as in production.
496 pub fn send(&self, notification: Notification) -> Delivery {
497 let watch = notification.watch();
498 let delivery = {
499 let mut state = lock(&self.shared.items);
500 // Before the new item, so a loss is reported at the point it happened
501 // rather than after changes that preceded it.
502 state.flush_latched();
503 let delivery = if state.free() > 0 {
504 state.queue.push_back(Entry::plain(notification));
505 Delivery::Queued
506 } else {
507 state.latch(watch);
508 Delivery::Latched
509 };
510 refresh_doorbell(&self.shared, &state);
511 delivery
512 };
513 self.shared.arrived.notify_all();
514 delivery
515 }
516
517 /// Claim one slot for a message that must not be lost.
518 ///
519 /// Take this *before* starting whatever produces the message (D-33): a
520 /// request reserves its completion slot at submit, an interactive
521 /// subscription its fault slot at registration. Backpressure then lands here,
522 /// on the caller's own thread, where it can be handled -- rather than at a
523 /// delivery that has no way to fail safely.
524 ///
525 /// Returns `None` if the queue is full. Dropping the reservation without
526 /// sending releases the slot.
527 pub fn reserve(&self) -> Option<Reservation> {
528 {
529 let mut state = lock(&self.shared.items);
530 if state.free() == 0 {
531 return None;
532 }
533 state.reserved += 1;
534 }
535 // Cloning takes the same lock, so it happens after the guard above is
536 // released. The clone is what keeps the queue connected for as long as
537 // the reservation is outstanding: a reservation can still deliver, so a
538 // receiver must not see the stream as finished while one exists.
539 Some(Reservation {
540 sender: self.clone(),
541 used: false,
542 })
543 }
544
545 /// Permanently reserve one slot for this subscription's fault question
546 /// (D-27/D-28).
547 ///
548 /// Unlike [`Sender::reserve`], the capacity this claims is never returned:
549 /// the same subscription may fault many times over its life, and
550 /// [`StandingSlot::send`] must never be able to fail. That is sound because
551 /// a watcher cannot be faulted twice concurrently (D-28) -- the state
552 /// machine cannot ask a second question before the first is answered -- so
553 /// one permanently-reserved slot is always enough, unlike
554 /// [`Sender::reserve`]'s slot, which is claimed and released per message.
555 ///
556 /// Returns `None` if the queue has no free capacity to carve out at
557 /// registration.
558 pub(crate) fn reserve_standing(&self) -> Option<StandingSlot> {
559 {
560 let mut state = lock(&self.shared.items);
561 if state.free() == 0 {
562 return None;
563 }
564 state.reserved += 1;
565 }
566 Some(StandingSlot {
567 sender: self.clone(),
568 state: Arc::new(Mutex::new(StandingState {
569 slot_alive: true,
570 in_flight: false,
571 queued: None,
572 })),
573 })
574 }
575
576 /// Whether a best-effort notification would be accepted right now.
577 ///
578 /// The observation tier checks this *before* arming a read rather than
579 /// discovering it at the enqueue, which is what turns saturation into a
580 /// grace period in the kernel's own change buffer instead of a loss (D-29).
581 ///
582 /// Accounts for a pending latched loss: [`Sender::send`] always flushes
583 /// every owed [`Delivery::Latched`] desync into the queue *before*
584 /// considering the caller's own notification, so a freed slot with a
585 /// latch still outstanding is not actually available to it. Without this,
586 /// a caller could see `true` immediately after a slot freed up, then
587 /// still get `Delivery::Latched` back from the very next `send` -- the
588 /// flush consumed the slot this check saw.
589 pub fn has_room(&self) -> bool {
590 lock(&self.shared.items).best_effort_room() > 0
591 }
592
593 /// Ask to be prodded when a saturated queue frees a slot.
594 ///
595 /// Registering is idempotent in effect: a spurious prod costs a producer only
596 /// a re-check.
597 pub(crate) fn register_resume(&self, who: &Arc<impl Resume + 'static>) {
598 lock(&self.shared.items)
599 .resumers
600 .push(Arc::downgrade(who) as Weak<dyn Resume>);
601 }
602}
603
604/// A claimed slot in the notification queue.
605///
606/// Holding one is what makes delivery infallible, so a control message is
607/// produced only once its room is already secured (D-33). Releasing it without
608/// sending is fine -- the slot returns to the pool -- and is what happens when a
609/// request is abandoned before it completes.
610pub struct Reservation {
611 sender: Sender,
612 used: bool,
613}
614
615impl Reservation {
616 /// Deliver the notification into the reserved slot.
617 ///
618 /// Cannot fail: the room was taken when this reservation was made.
619 pub fn send(mut self, notification: Notification) {
620 {
621 let mut state = lock(&self.sender.shared.items);
622 // Flushed while this reservation is still counted, so the latch can
623 // only take genuinely free slots and never this one.
624 state.flush_latched();
625 state.reserved -= 1;
626 state.queue.push_back(Entry::plain(notification));
627 refresh_doorbell(&self.sender.shared, &state);
628 }
629 self.used = true;
630 self.sender.shared.arrived.notify_all();
631 }
632}
633
634impl Drop for Reservation {
635 fn drop(&mut self) {
636 if !self.used {
637 let resumers = {
638 let mut state = lock(&self.sender.shared.items);
639 state.reserved -= 1;
640 // A dropped-unused reservation frees capacity exactly like a
641 // drained item does, so a producer paused in `Backpressured`
642 // (D-29) must be prodded the same way, or it can stay paused
643 // forever waiting on a receive that was never going to come.
644 freed_resumers(&mut state, true)
645 };
646 prod(resumers);
647 }
648 }
649}
650
651/// State shared between a [`StandingSlot`] and every [`StandingHold`] it has
652/// ever handed to a queued entry, protected by its own lock so the two halves
653/// of a reservation's transfer -- whether the slot is still alive, and
654/// whether its last send is still queued -- are always read and mutated
655/// together with the `reserved` count they gate, never observed mid-way.
656///
657/// Lock order is fixed as *the queue's `items`, then this* everywhere it is
658/// taken (`send`, `StandingSlot::drop`, and [`take`]), which is what makes the
659/// combined transition atomic: whichever of `StandingSlot::drop` or `take` runs
660/// first holds this lock for its entire mutation of `reserved`, so the other
661/// side cannot observe a half-finished transfer. `StandingHold::drop` takes
662/// neither lock -- see that impl for why it must not.
663struct StandingState {
664 /// Whether the owning `StandingSlot` has been dropped. Once false, no
665 /// further send can ever happen, so an outstanding hold's eventual drain
666 /// must simply let its transferred unit go free rather than handing it
667 /// back to a slot that no longer exists to reuse it.
668 slot_alive: bool,
669 /// Whether the last-sent message is still queued, undrained -- meaning
670 /// the unit of `reserved` capacity this slot carved out is currently
671 /// manifested as that queued entry rather than sitting in `reserved`.
672 in_flight: bool,
673 /// The queued entry's current notification value while `in_flight` is
674 /// true (PR #20 review response); `None` otherwise. A second `send`
675 /// while `in_flight` overwrites this in place -- reachable when an
676 /// interactive watch is answered before its queued question is drained
677 /// and the retry fails again -- rather than decrementing `reserved` a
678 /// second time for a slot that only ever carves out one unit.
679 queued: Option<Notification>,
680}
681
682/// A permanent slot in the notification queue, reserved once and reused for as
683/// long as the subscription lives (D-27/D-28).
684///
685/// See [`Sender::reserve_standing`] for why one slot is always enough. Unlike
686/// [`Reservation`], sending through this does not release the reservation --
687/// there is nothing to release until the subscription itself ends, at which
688/// point `Drop` returns the slot to the pool.
689pub(crate) struct StandingSlot {
690 sender: Sender,
691 state: Arc<Mutex<StandingState>>,
692}
693
694impl StandingSlot {
695 /// Deliver the notification. Cannot fail: the slot was carved out of
696 /// capacity at registration and is never returned while this is held.
697 ///
698 /// Coalesces in place with a previous send whose entry is still queued
699 /// (undrained) rather than pushing a second entry (PR #20 review
700 /// response): the slot's single carved-out unit of `reserved` capacity
701 /// must never be double-spent, and it is exactly this second-send-while-
702 /// undrained case that would otherwise underflow it.
703 ///
704 /// Lock order: `items`, then this slot's own `standing` state -- the
705 /// same order [`take`] and [`StandingSlot::drop`] use, so none of the
706 /// three can ever deadlock waiting on each other.
707 pub(crate) fn send(&self, notification: Notification) {
708 {
709 let mut state = lock(&self.sender.shared.items);
710 let mut standing = lock(&self.state);
711 if standing.in_flight {
712 standing.queued = Some(notification);
713 } else {
714 standing.in_flight = true;
715 standing.queued = Some(notification);
716 drop(standing);
717 state.flush_latched();
718 // The permanent reservation transfers into the queued entry
719 // (`Entry::standing`) for as long as it sits there, rather
720 // than staying counted in `reserved` on top of the entry's
721 // own `queue.len()` slot -- `take` restores it atomically
722 // with the pop when the entry is eventually drained.
723 state.reserved -= 1;
724 let hold = StandingHold {
725 // Weak, not Arc: this hold lives inside `Entry` inside
726 // `state.queue` inside this very `Shared` (PR #20 review
727 // response) -- a strong reference here would be a
728 // self-cycle that keeps `Shared` (and its doorbell
729 // handle) alive forever if this entry is ever left
730 // permanently undrained.
731 shared: Arc::downgrade(&self.sender.shared),
732 state: Arc::clone(&self.state),
733 resolved: false,
734 };
735 state.queue.push_back(Entry::standing(hold));
736 refresh_doorbell(&self.sender.shared, &state);
737 }
738 }
739 self.sender.shared.arrived.notify_all();
740 }
741}
742
743impl Drop for StandingSlot {
744 /// Lock order: `items`, then `standing` -- matching [`StandingSlot::send`]
745 /// and [`take`] (PR #20 review response), so this can never deadlock
746 /// against either.
747 fn drop(&mut self) {
748 let mut state = lock(&self.sender.shared.items);
749 let mut standing = lock(&self.state);
750 standing.slot_alive = false;
751 if standing.in_flight {
752 // The outstanding hold -- or `take`, inline with the pop -- owns
753 // releasing this reservation once its message drains; finding
754 // the slot already gone, it lets the capacity go free rather
755 // than handing it back here, so this slot has nothing left to
756 // release itself.
757 return;
758 }
759 drop(standing);
760 // As for `Reservation`'s drop: releasing this permanent slot frees
761 // capacity without any item being drained, so a paused producer must
762 // still be prodded on the same zero-to-one transition.
763 state.reserved -= 1;
764 let resumers = freed_resumers(&mut state, true);
765 drop(state);
766 prod(resumers);
767 }
768}
769
770impl std::fmt::Debug for StandingSlot {
771 fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
772 f.debug_struct("StandingSlot").finish_non_exhaustive()
773 }
774}
775
776/// The record that a [`StandingSlot::send`] has transferred its permanent
777/// reservation into a queued entry.
778///
779/// The release itself belongs to [`take`], performed inline with the pop under
780/// the same `items` lock (PR #20 review response): popping a standing entry
781/// exposed its queue slot before the reservation was restored under the old
782/// design -- where this type's own `Drop` did the restoring, deferred until
783/// after the queue lock was released -- and a producer woken in that gap could
784/// overcommit capacity. `take` therefore performs the release itself and marks
785/// `resolved`. What is left here is the record of the transfer, which is what
786/// [`StandingSlot::drop`] reads to tell whether it still owns its unit; it is
787/// not a second release path, and this type's `Drop` explains why no discard may
788/// delegate the release back to it.
789struct StandingHold {
790 /// `Weak`, not `Arc` (PR #20 review response): this hold lives inside an
791 /// `Entry` inside `Shared.items.queue` -- inside the very `Shared` a
792 /// strong reference here would point back to, a self-cycle that would
793 /// keep `Shared` (and its doorbell handle) alive forever if this entry
794 /// is ever left permanently undrained (every `Sender`/`Receiver` and
795 /// route dropped before it is). Upgrading fails only when `Shared`
796 /// itself is already being torn down, in which case there is no
797 /// `reserved` accounting left to update.
798 shared: Weak<Shared>,
799 state: Arc<Mutex<StandingState>>,
800 /// Set once [`take`] has performed this hold's release inline with the pop.
801 /// Every hold that leaves the queue leaves it that way, so this is the arm
802 /// `Drop` actually takes; see `Drop` for why the other arm is a tripwire
803 /// rather than a fallback.
804 resolved: bool,
805}
806
807impl Drop for StandingHold {
808 /// A tripwire, not a release path.
809 ///
810 /// Both arms below return, and that is the whole behaviour: a hold is
811 /// reachable only from `state.queue`, the one site that removes an entry
812 /// from that queue ([`take`]) settles the reservation itself and sets
813 /// `resolved`, and the only other way a hold dies is `Shared` being torn
814 /// down, where `upgrade` fails and there is no accounting left to update.
815 ///
816 /// It deliberately does **not** restore the reservation itself, and a future
817 /// discard path must not make it. `take` receives `&mut State`, so its caller
818 /// holds the `items` guard -- and any other way to remove an entry from
819 /// `state.queue` needs that same guard, so a hold discarded on such a path is
820 /// dropped *inside* the lock. Re-taking it here would deadlock, because
821 /// [`lock`] is a plain non-reentrant `Mutex::lock`. That is measured, not
822 /// supposed: with a forced unwind out of `take`, the earlier body that did
823 /// the accounting here hung indefinitely, while the same unwind with this
824 /// `Drop` short-circuited failed immediately. **Any discard path must release
825 /// the reservation under the lock it already holds, exactly as [`take`]
826 /// does.**
827 fn drop(&mut self) {
828 if self.resolved {
829 return;
830 }
831 if self.shared.upgrade().is_none() {
832 // `Shared` is already being torn down; nothing to update.
833 return;
834 }
835 // Reaching here means a hold outlived its entry while the queue was
836 // still alive. Today the only way that can happen is an unwind out of
837 // `take` between the pop and `resolved` being set, and there the original
838 // panic is the real diagnostic -- so it is left to propagate rather than
839 // replaced by an abort from a second panic. Anything else is a new
840 // discard path that has not settled its reservation; see above for the
841 // one way it is allowed to.
842 // **`assert!`, not `debug_assert!`.** A tripwire compiled out of release
843 // builds is not a tripwire: the violation this guards is a future
844 // discard path that forgets to settle its reservation, which would
845 // leave the capacity accounting corrupted and, in release, silently.
846 //
847 // The "no second panic during an unwind" property is preserved by the
848 // predicate rather than by the assertion's flavour: when the thread is
849 // already panicking this passes, so the original panic -- the real
850 // diagnostic -- propagates untouched. It fires only when the hold
851 // outlived its entry *without* an unwind, which is the case that has no
852 // other explanation.
853 assert!(
854 std::thread::panicking(),
855 "a StandingHold outlived its entry with the queue still alive: a \
856 discard path must release the reservation under the `items` lock it \
857 already holds, as `take` does"
858 );
859 }
860}
861
862impl std::fmt::Debug for Reservation {
863 fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
864 f.debug_struct("Reservation").finish_non_exhaustive()
865 }
866}
867
868impl Drop for Sender {
869 fn drop(&mut self) {
870 let mut state = lock(&self.shared.items);
871 state.senders -= 1;
872 let last = state.senders == 0;
873 if last {
874 // The end of the stream is something to observe, so the doorbell
875 // reports it too -- otherwise a client waiting on it would wait for a
876 // notification nothing can send.
877 refresh_doorbell(&self.shared, &state);
878 }
879 drop(state);
880 if last {
881 // Wake anyone blocked in `recv`, so a queue that can never be filled
882 // again does not hang its receiver.
883 self.shared.arrived.notify_all();
884 }
885 }
886}
887
888impl std::fmt::Debug for Sender {
889 fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
890 f.debug_struct("Sender").finish_non_exhaustive()
891 }
892}
893
894/// The client-side half: the only way to observe notifications.
895pub struct Receiver {
896 shared: Arc<Shared>,
897}
898
899impl Receiver {
900 /// Take the next notification if one is already available.
901 #[must_use]
902 pub fn try_recv(&self) -> Option<Notification> {
903 let (entry, resumers) = {
904 let mut state = lock(&self.shared.items);
905 let taken = take(&mut state);
906 refresh_doorbell(&self.shared, &state);
907 let resumers = freed_resumers(&mut state, taken.is_some());
908 (taken, resumers)
909 };
910 // `entry` (and any `StandingHold` it carries) is dropped here, after
911 // the queue lock above has already been released -- a hold's own
912 // `Drop` re-locks the same mutex to restore its reservation, which
913 // would deadlock if it ran while that lock was still held.
914 prod(resumers);
915 entry.map(Entry::into_notification)
916 }
917
918 /// Block until a notification is available, or every sender is gone.
919 ///
920 /// Returns `None` only when nothing is queued, nothing is latched, *and* no
921 /// sender remains, so a client loop terminates on teardown instead of
922 /// hanging -- and never before it has been told about a loss.
923 #[must_use]
924 pub fn recv(&self) -> Option<Notification> {
925 let mut state = lock(&self.shared.items);
926 loop {
927 if let Some(entry) = take(&mut state) {
928 refresh_doorbell(&self.shared, &state);
929 let resumers = freed_resumers(&mut state, true);
930 drop(state);
931 prod(resumers);
932 // As in `try_recv`: `entry` is dropped only after the lock
933 // above is released.
934 return Some(entry.into_notification());
935 }
936 if state.senders == 0 {
937 return None;
938 }
939 state = self
940 .shared
941 .arrived
942 .wait(state)
943 .unwrap_or_else(|poison| poison.into_inner());
944 }
945 }
946
947 /// Block for at most `timeout`.
948 ///
949 /// Returns `None` on timeout as well as on teardown; a caller that must tell
950 /// them apart can check [`Receiver::is_disconnected`].
951 #[must_use]
952 pub fn recv_timeout(&self, timeout: Duration) -> Option<Notification> {
953 let deadline = Instant::now() + timeout;
954 let mut state = lock(&self.shared.items);
955 loop {
956 if let Some(entry) = take(&mut state) {
957 refresh_doorbell(&self.shared, &state);
958 let resumers = freed_resumers(&mut state, true);
959 drop(state);
960 prod(resumers);
961 // As in `try_recv`: `entry` is dropped only after the lock
962 // above is released.
963 return Some(entry.into_notification());
964 }
965 if state.senders == 0 {
966 return None;
967 }
968 let remaining = deadline.checked_duration_since(Instant::now())?;
969 let (next, _) = self
970 .shared
971 .arrived
972 .wait_timeout(state, remaining)
973 .unwrap_or_else(|poison| poison.into_inner());
974 state = next;
975 }
976 }
977
978 /// A manual-reset event that is signalled whenever this receiver has
979 /// something to take.
980 ///
981 /// This is what lets a client integrate with its own thread pool instead of
982 /// dedicating a thread to a blocking [`Receiver::recv`]: wait on the handle,
983 /// then drain with [`Receiver::try_recv`] until it yields `None`.
984 ///
985 /// The event is created on the first call, so a client that never asks for it
986 /// pays for no kernel object. It stays signalled while anything is
987 /// outstanding, including once the stream has ended -- so a waiter learns
988 /// about disconnection rather than waiting for a notification that can never
989 /// arrive.
990 ///
991 /// The borrow is deliberate: the event belongs to this queue and must not be
992 /// closed by a caller. Use [`Receiver::doorbell_shared`] or
993 /// [`Receiver::doorbell_owned`] where ownership is required, such as arming a
994 /// `ThreadpoolWait`.
995 ///
996 /// # Errors
997 ///
998 /// Returns the error from `CreateEventW` on the first call.
999 pub fn doorbell(&self) -> io::Result<BorrowedHandle<'_>> {
1000 self.ensure_doorbell()?;
1001 Ok(self
1002 .shared
1003 .doorbell
1004 .get()
1005 .expect("the doorbell was just created")
1006 .as_handle())
1007 }
1008
1009 /// A duplicate of [`Receiver::doorbell`] that the caller owns.
1010 ///
1011 /// The duplicate refers to the same event, so signalling reaches both; the
1012 /// caller closes its own copy whenever it likes. This is the form a
1013 /// `ThreadpoolWait` needs, since arming one takes ownership of its target.
1014 ///
1015 /// # Errors
1016 ///
1017 /// Returns the error from `CreateEventW` or `DuplicateHandle`.
1018 pub fn doorbell_owned(&self) -> io::Result<OwnedHandle> {
1019 duplicate_handle(
1020 self.doorbell()?,
1021 DuplicateAccess::Same,
1022 Inheritance::NotInheritable,
1023 )
1024 }
1025
1026 /// The doorbell as a shared handle: the queue's own event, held by one more
1027 /// owner.
1028 ///
1029 /// This is [`Receiver::doorbell`] with the ownership the borrow lacks, and no
1030 /// new kernel handle: the result is a clone of the handle the queue holds, so
1031 /// its raw handle is the borrow's, and the event closes when the last holder
1032 /// of it, the queue's own included, drops. A `ThreadpoolWait` can take it
1033 /// through `WaitableHandle::assume_waitable_shared`, which is what arming one
1034 /// needs without a duplicate per waiter.
1035 ///
1036 /// The queue keeps the event in agreement with its contents only for as long as
1037 /// the queue exists. Holding this does not keep the queue alive, and nothing
1038 /// signals or resets the event once the queue is gone, so a holder that outlives
1039 /// every sender and the receiver is waiting on an event nothing will change. Use
1040 /// [`Receiver::doorbell_owned`] for a handle that is the caller's own and can be
1041 /// closed whenever the caller likes.
1042 ///
1043 /// # Errors
1044 ///
1045 /// Returns the error from `CreateEventW` on the first call to any doorbell
1046 /// method.
1047 pub fn doorbell_shared(&self) -> io::Result<SharedHandle> {
1048 self.ensure_doorbell()?;
1049 Ok(self
1050 .shared
1051 .doorbell
1052 .get()
1053 .expect("the doorbell was just created")
1054 .clone())
1055 }
1056
1057 /// Create the doorbell if it does not exist yet.
1058 fn ensure_doorbell(&self) -> io::Result<()> {
1059 if self.shared.doorbell.get().is_some() {
1060 return Ok(());
1061 }
1062 // Created under the queue lock so its initial state cannot disagree with
1063 // the queue it reports on: a client that asks for a doorbell after
1064 // notifications have already arrived must find it signalled.
1065 let state = lock(&self.shared.items);
1066 if self.shared.doorbell.get().is_none() {
1067 let event = create_event(state.pending())?;
1068 let _ = self.shared.doorbell.set(SharedHandle::new(event));
1069 }
1070 Ok(())
1071 }
1072
1073 /// Whether every sender has been dropped, so nothing further can arrive.
1074 #[must_use]
1075 pub fn is_disconnected(&self) -> bool {
1076 lock(&self.shared.items).senders == 0
1077 }
1078
1079 /// How many notifications are queued right now.
1080 ///
1081 /// Excludes latched losses, which are not queued -- that is the point of the
1082 /// latch. [`Receiver::latched`] counts those.
1083 ///
1084 /// **This is not "how much is there to take".** A drained queue with a loss
1085 /// still owed reports `0` here while [`Receiver::recv`] would still yield a
1086 /// synthesised `Desync { QueueFull }`. Use [`Receiver::has_pending`] to ask
1087 /// whether anything is collectable.
1088 #[must_use]
1089 pub fn len(&self) -> usize {
1090 lock(&self.shared.items).queue.len()
1091 }
1092
1093 /// Whether nothing is queued right now.
1094 ///
1095 /// **Not the same as "nothing to collect"**, and the difference is a live
1096 /// hazard rather than a nicety: this excludes owed latched losses and the
1097 /// end of the stream, both of which [`Receiver::recv`] still reports. A
1098 /// drain loop written as `while !receiver.is_empty()` therefore exits with a
1099 /// `Desync { QueueFull }` still owed -- and, because the doorbell stays
1100 /// signalled while a loss is owed (D-41), a client that waits on the
1101 /// doorbell and then tests this spins without ever collecting the report it
1102 /// was woken for. Use [`Receiver::has_pending`], which is the predicate the
1103 /// doorbell is actually signalled on.
1104 #[must_use]
1105 pub fn is_empty(&self) -> bool {
1106 self.len() == 0
1107 }
1108
1109 /// Whether a [`Receiver::recv`] would produce something rather than block.
1110 ///
1111 /// This is *the* predicate the doorbell is signalled on (D-41), so it is the
1112 /// one to test after a doorbell wait. "Something to take" is deliberately
1113 /// three things, not one:
1114 ///
1115 /// - a queued notification ([`Receiver::len`]);
1116 /// - an **owed latched loss** ([`Receiver::latched`]) -- not in the queue,
1117 /// but still collectable, and synthesised on demand (D-39);
1118 /// - the **end of the stream** ([`Receiver::is_disconnected`]) -- a client
1119 /// must learn the stream ended rather than wait for a notification
1120 /// nothing can send.
1121 ///
1122 /// Prefer this over [`Receiver::is_empty`] anywhere the question is "is
1123 /// there anything for me?" rather than "how deep is the queue?".
1124 #[must_use]
1125 pub fn has_pending(&self) -> bool {
1126 lock(&self.shared.items).pending()
1127 }
1128
1129 /// How many subscriptions are owed a `Desync { QueueFull }`.
1130 #[must_use]
1131 pub fn latched(&self) -> usize {
1132 lock(&self.shared.items).latched.len()
1133 }
1134
1135 /// The queue's bound.
1136 #[must_use]
1137 pub fn capacity(&self) -> usize {
1138 lock(&self.shared.items).capacity
1139 }
1140}
1141
1142/// Take the next item: a queued notification, or -- once the queue is drained --
1143/// a synthesised report of a loss that never fitted.
1144///
1145/// The queue comes first because the latch records a loss that happened *after*
1146/// everything still queued: the queue was full at the time, so surfacing the
1147/// desync early would claim the hole is older than it is and break the ordering
1148/// D-12 promises.
1149/// Take the next item: a queued entry, or -- once the queue is drained -- a
1150/// synthesised report of a loss that never fitted.
1151///
1152/// The queue comes first because the latch records a loss that happened *after*
1153/// everything still queued: the queue was full at the time, so surfacing the
1154/// desync early would claim the hole is older than it is and break the ordering
1155/// D-12 promises.
1156///
1157/// A standing entry's reservation is restored right here, inline with the pop,
1158/// under the same `items` lock `state` already holds (PR #20 review response):
1159/// popping the entry first and restoring the reservation later (in the
1160/// `StandingHold`'s own deferred `Drop`) left a window where `queue.len()` had
1161/// already dropped but `reserved` had not yet gone back up, so `free()`
1162/// over-reported room by one and a producer woken in that gap could overcommit
1163/// capacity. The returned `Entry` still carries its (now inert) `StandingHold`
1164/// so a caller may defer *dropping* it until after the queue lock is released
1165/// -- the hold's own `Drop` is a no-op here, since `resolved` is already set --
1166/// but the reservation itself is never left outstanding past this call. Setting
1167/// `resolved` is not bookkeeping that can be skipped: it is what keeps the hold
1168/// off the tripwire in `StandingHold::drop`, which any future path that removes
1169/// an entry from the queue must satisfy the same way -- by releasing inline,
1170/// here, under the lock it already holds.
1171fn take(state: &mut State) -> Option<Entry> {
1172 if let Some(mut entry) = state.queue.pop_front() {
1173 if matches!(entry.notification, EntryNotification::Standing) {
1174 let hold = entry
1175 ._standing_hold
1176 .as_mut()
1177 .expect("a Standing entry always carries a StandingHold");
1178 let notification = {
1179 let mut standing = lock(&hold.state);
1180 standing.in_flight = false;
1181 let notification = standing.queued.take().expect(
1182 "a Standing entry's queued value is set for as long as it is undrained",
1183 );
1184 if standing.slot_alive {
1185 state.reserved += 1;
1186 }
1187 notification
1188 };
1189 hold.resolved = true;
1190 entry.notification = EntryNotification::Plain(notification);
1191 }
1192 return Some(entry);
1193 }
1194 state.latched.pop_front().map(|watch| {
1195 Entry::plain(Notification::Desync {
1196 watch,
1197 cause: DesyncCause::QueueFull,
1198 })
1199 })
1200}
1201
1202impl std::fmt::Debug for Receiver {
1203 fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
1204 let state = lock(&self.shared.items);
1205 f.debug_struct("Receiver")
1206 .field("queued", &state.queue.len())
1207 .field("capacity", &state.capacity)
1208 .field("latched", &state.latched.len())
1209 .field("disconnected", &(state.senders == 0))
1210 .finish_non_exhaustive()
1211 }
1212}
1213
1214/// Create a connected sender/receiver pair with the default bound.
1215///
1216/// A client reaches a queue through [`Monitor::session`](crate::monitor::Monitor::session),
1217/// which chooses the bound; this exists for tests that need a queue without one.
1218#[cfg(test)]
1219pub(crate) fn channel() -> (Sender, Receiver) {
1220 channel_with_bound(DEFAULT_BOUND)
1221}
1222
1223/// Create a connected sender/receiver pair holding at most `bound` notifications.
1224///
1225/// The bound is a [`NonZeroUsize`] rather than a checked `usize` because a zero
1226/// bound is not a runtime condition to report -- it is a queue that could not
1227/// carry even the desync announcing its own saturation, making the crate's
1228/// never-silently-lose guarantee vacuous. Making it unrepresentable rejects it at
1229/// construction, where D-11 asks for it, and leaves no error path to handle.
1230///
1231/// This is also the consumer test seam (D-81/D-82), reachable by a downstream
1232/// consumer only under the `test-util` feature: a consumer that wants to test its
1233/// own notification-handling code constructs a pair here, pushes synthetic
1234/// notifications through the [`Sender`] with [`Sender::send`], and drains the
1235/// [`Receiver`] exactly as it would one from
1236/// [`Monitor::session`](crate::Monitor::session) -- with no real filesystem or
1237/// thread pool. See the crate-level "Testing your consumer code".
1238pub fn channel_with_bound(bound: NonZeroUsize) -> (Sender, Receiver) {
1239 let shared = Arc::new(Shared {
1240 items: Mutex::new(State {
1241 queue: VecDeque::new(),
1242 capacity: bound.get(),
1243 reserved: 0,
1244 latched: VecDeque::new(),
1245 senders: 1,
1246 resumers: Vec::new(),
1247 }),
1248 arrived: Condvar::new(),
1249 doorbell: OnceLock::new(),
1250 });
1251 (
1252 Sender {
1253 shared: Arc::clone(&shared),
1254 },
1255 Receiver { shared },
1256 )
1257}
1258
1259/// The producers to prod, if taking an item has just made room.
1260///
1261/// Returns them rather than calling them, because a producer's `resume` must not
1262/// run under the queue lock: the obvious implementation re-arms a read, and a
1263/// failed arm reports itself by *sending*, which would take this very lock.
1264///
1265/// Dead producers are pruned here, which is the only place the list is walked.
1266fn freed_resumers(state: &mut State, took_one: bool) -> Vec<Arc<dyn Resume>> {
1267 // Only on the transition into having room: prodding on every take would put
1268 // a wake behind every single notification a client drains. The edge is
1269 // expressed in `best_effort_room` -- the same quantity `Sender::has_room`
1270 // tests -- so the prod fires exactly when a parked producer's re-check will
1271 // now succeed. Every step that frees capacity (taking a queued item,
1272 // synthesising a latched one, releasing a reservation) raises it by exactly
1273 // one, so `== 1` catches the crossing and cannot be stepped over.
1274 if !took_one || state.resumers.is_empty() || state.best_effort_room() != 1 {
1275 return Vec::new();
1276 }
1277 let mut live = Vec::with_capacity(state.resumers.len());
1278 let mut awake = Vec::new();
1279 for weak in state.resumers.drain(..) {
1280 if let Some(producer) = weak.upgrade() {
1281 live.push(Arc::downgrade(&producer));
1282 awake.push(producer);
1283 }
1284 }
1285 state.resumers = live;
1286 awake
1287}
1288
1289/// Tell each producer there may be room now.
1290fn prod(resumers: Vec<Arc<dyn Resume>>) {
1291 for producer in resumers {
1292 producer.resume();
1293 }
1294}
1295
1296/// Lock, recovering the guard if a previous holder panicked.
1297///
1298/// A poisoned lock here means some thread panicked while holding it; the queue is
1299/// a plain `VecDeque` plus a count, both of which are left structurally intact by
1300/// every path, so refusing to proceed would strand the receiver rather than
1301/// protect anything.
1302fn lock<T>(mutex: &Mutex<T>) -> std::sync::MutexGuard<'_, T> {
1303 mutex.lock().unwrap_or_else(|poison| poison.into_inner())
1304}
1305
1306/// Bring the doorbell into agreement with the state.
1307///
1308/// Every mutation ends here, under the same lock a receiver holds while deciding
1309/// there is nothing to take -- which is what makes a lost wakeup impossible
1310/// rather than merely unlikely: there is no window between the two decisions for
1311/// one to fall into.
1312fn refresh_doorbell(shared: &Shared, state: &State) {
1313 let Some(doorbell) = shared.doorbell.get() else {
1314 // Never asked for, so there is nothing to keep in agreement.
1315 return;
1316 };
1317 let raw = doorbell.as_raw_handle();
1318 // SAFETY: `raw` is a live manual-reset event owned by this queue for as long
1319 // as the queue exists, and neither call has any other precondition.
1320 unsafe {
1321 if state.pending() {
1322 SetEvent(raw);
1323 } else {
1324 ResetEvent(raw);
1325 }
1326 }
1327}
1328
1329/// Create a manual-reset event in the given initial state.
1330fn create_event(signalled: bool) -> io::Result<OwnedHandle> {
1331 // SAFETY: creates an unnamed event with default security attributes; both
1332 // pointer arguments are null by design.
1333 let raw = unsafe {
1334 CreateEventW(
1335 ptr::null(),
1336 TRUE,
1337 if signalled { TRUE } else { FALSE },
1338 ptr::null(),
1339 )
1340 };
1341 if raw.is_null() {
1342 return Err(io::Error::last_os_error());
1343 }
1344 // SAFETY: the call returned a fresh, exclusively owned event handle.
1345 Ok(unsafe { OwnedHandle::from_raw_handle(raw) })
1346}
1347
1348#[cfg(test)]
1349mod tests;