pub struct IoRing<T = (), X = (), K = Open> { /* private fields */ }Expand description
An owned IoRing, closed with CloseIoRing on drop.
Not Clone: cloning would give two owners of the same native ring, and
CloseIoRing would run twice. Not Sync: building a submission is not
thread-safe (D-5 in DESIGN-NOTES.md), so sharing &IoRing across
threads is deliberately not offered – a consumer wanting concurrent
access chooses a delivery architecture (M4 / M6+) rather than relying on
this type to serialize for them.
Implementations§
Source§impl<T, X, K> IoRing<T, X, K>
impl<T, X, K> IoRing<T, X, K>
Sourcepub fn version(&self) -> RingVersion
pub fn version(&self) -> RingVersion
The version this ring was created at.
Sourcepub fn supports(&self, op: Op) -> bool
pub fn supports(&self, op: Op) -> bool
Whether this ring supports op, from the capability set cached at
construction.
Answers what the kernel’s op table contains, not what this crate’s
safe push surface reaches (M10.1, Op’s own docs list the mapping).
The two coincide for every op except Op::Nop, which has no
crate::Batch method at all: a nop owns no buffer, so there is
nothing for the ring to hand back, and it is reachable only
through IoRing::push_raw. A true here therefore means “the
kernel would accept this op”, not “a Batch method exists to push
it”.
Sourcepub fn completion_event(&mut self) -> Result<OwnedHandle>
pub fn completion_event(&mut self) -> Result<OwnedHandle>
An owned duplicate of this ring’s completion event, so a caller can wait on the ring alongside other handles without surrendering it (M11.1, D-20).
The ring creates the event, attaches it with
SetIoRingCompletionEvent, keeps ownership, and hands back a
duplicate. Closing the returned handle is therefore always safe: the
ring keeps signalling its own copy, and a DuplicateHandle’d event is
still signalled after the original is closed.
Idempotent. Repeat calls return another duplicate of the same event rather than attaching a new one, so two subsystems can each ask without silently detaching the other’s.
This is not a third delivery architecture – it is Model B with a
multiplexed wakeup source, changing only what the domain thread blocks
on, never who owns, submits, or drains (Batch::submit_and_wait is
still the single-source shape). Use it when ring I/O has to be waited
on alongside non-ring I/O, which
IOSQE_FLAGS_DRAIN_PRECEDING_OPS cannot order across.
§The event is an edge, not a level
Measured, not inferred, and getting it wrong hangs rather than just slowing down (D-19). The event is signalled when the completion queue transitions from empty to non-empty. It is not signalled once per completion, and it is not level-triggered: eight completions arriving at once into an empty queue produce exactly one wakeup, and a completion arriving into an already-non-empty queue produces none.
Two rules follow, and they are contract rather than advice:
- Drain to empty before waiting again –
IoRing::try_popuntil it yieldsNone, on every pass through a multiplexed wait loop, not only the pass where this handle signalled. A wait entered with entries still in the queue blocks until some later completion arrives after the queue has been emptied, which may be never. That is a lost-wakeup deadlock, not a latency wobble. - A wake with nothing to pop is normal. It must not be treated as an error or as a spurious wakeup. This method deliberately produces one: the event is signalled once before it returns, so a caller who had already submitted work never misses a backlog that arrived before the event was attached.
The event is auto-reset, and exactly one waiter per ring is supported (D-21): the drain that restores the empty state, and so re-arms the edge, must run to empty exactly once. Two threads waiting on one ring’s event cannot be made correct.
§If you are arming a thread-pool wait on this handle
SetThreadpoolWait documents that “you must re-register the event
with the wait object before signaling it each time to trigger the wait
callback”. This method signals the event before it returns (rule 2
above), so by the time you have a handle to build a wait object
around, that setup signal has already happened – in the order the
rule forbids, and so with no documented guarantee that your callback
runs. Combined with the edge rule above, a ring whose queue never
returns to empty has no second wakeup coming, so a setup signal that
does not reach your callback strands the backlog permanently rather
than merely delaying it.
The remedy needs nothing this method does not already give you – after arming the wait, signal your own duplicate yourself: ///
let event = ring.completion_event()?;
// ... build the wait object around `event`, then arm it ...
// Only now raise the wakeup, in the documented order. A wake with
// nothing to pop is normal (rule 2), so this is always safe.
unsafe { windows_sys::Win32::System::Threading::SetEvent(event.as_raw_handle()) };EventDelivery already does this for you and
is the better answer if you do not need the handle itself.
examples/model_b_multiplexed.rs is this whole shape worked end to
end – a caller-owned ring waited on alongside a shutdown latch, with
the quiesce that shutdown-while-outstanding requires (M11.6).
§Errors
Returns io::ErrorKind::Unsupported if the running system does not
report IORING_FEATURE_SET_COMPLETION_EVENT; this crate refuses to
silently substitute a polling thread, since a caller who asked for an
event and got a spun-up thread has been told something false. Also
returns any error from CreateEventW,
SetIoRingCompletionEvent, SetEvent, or duplicating the handle.
Sourcepub fn supports_raw(&self, op_code: IORING_OP_CODE) -> bool
pub fn supports_raw(&self, op_code: IORING_OP_CODE) -> bool
Whether this ring supports a raw op code, including one this crate does not yet name (D-7).
Its reason to exist is an op outside Op, which by definition this
ring’s cached capability set was never probed for – but passing a
named op’s Op::code is equally in contract, and answers
identically to IoRing::supports (M10.1). The difference between
them is cost, not truth: supports is a bit test against the set
probed once at construction, this is an IsIoRingOpSupported call
every time.
What a true here does not mean is that the op became pushable: an
op outside Op has no builder method whatever this answers, so
IoRing::push_raw remains the only route to one.
Sourcepub fn registered_file_count(&self) -> u32
pub fn registered_file_count(&self) -> u32
How many file handles this ring has reserved for registration – not how many are confirmed registered (M5.1, M10.3, D-31).
The count advances the instant a BuildIoRingRegisterFileHandles
call queues, never when its completion is observed. Two consequences
a caller must not be surprised by:
- it is already advanced before any completion has been popped, so it
cannot be used to decide whether a registration has taken effect –
claim the completion with
crate::PendingFileRegistration::claim_iffor that; - it stays advanced after a registration whose completion reported
failure, which is why such a registration cannot be retried on
this ring (
crate::Batch::register_files).
Because a ring accepts at most one registration that assigns an
index, this is 0 until that registration is queued and its count
thereafter; there is no second registration for it to serve as a base
index for.
Sourcepub fn registered_buffer_count(&self) -> u32
pub fn registered_buffer_count(&self) -> u32
As IoRing::registered_file_count, for registered buffers (M5.2) –
a reserved count, not a confirmed one, with the same two
consequences (M10.3, D-31).
Sourcepub fn outstanding(&self) -> usize
pub fn outstanding(&self) -> usize
How many operations this ring believes are still outstanding: minted
(via reserve_user_data) but not yet observed to have completed (via
record_completion, which retires only an identity this ring minted).
Sourcepub unsafe fn push_raw(
&mut self,
build: impl FnOnce(*mut c_void, usize) -> HRESULT,
) -> Result<usize>where
K: AllowsRaw,
pub unsafe fn push_raw(
&mut self,
build: impl FnOnce(*mut c_void, usize) -> HRESULT,
) -> Result<usize>where
K: AllowsRaw,
Queue a raw, not-yet-wrapped SQE via a caller-supplied Build* call
(M3.5, D-7).
build receives this ring’s native handle and a freshly reserved
UserData value, and must call exactly one BuildIoRing* function
with them, returning its HRESULT. On success the UserData is
returned so the caller can match it against a later Completion
popped by IoRing::try_pop; on failure the reservation is
released, since the op was never actually queued.
Matching by hand is correct here, and only here. This seam creates
no inventory entry – it cannot, since it does not know what the
caller’s build closure queued or what it needs held – so a pop for
this operation reports None for what the ring was holding. That is
the legitimate half of D-75’s two causes,
and choosing this seam is what makes it legitimate. Every other push
hands its payload to the ring and gets it back from the pop.
§Errors
Returns any error build’s HRESULT reports.
§Safety
build must queue an SQE for a self-contained op: everything the
kernel reads or writes for it must stay valid until the
corresponding completion is observed, and this crate cannot verify
what build does with the handle it is given. This is the same
framing as windows-overlapped-io-sys’s raw ioctl seam
(device.rs): the mechanics of building an SQE need nothing unsafe,
but this crate cannot audit an arbitrary Build* call, so the seam
itself is unsafe.
build must also queue its SQE with exactly the user_data it was
handed, and queue only one. A completion carrying any other identity
is one this ring never minted, and the pop that receives it panics
(D-79).
Sourcepub fn run_down(&mut self) -> Result<()>
pub fn run_down(&mut self) -> Result<()>
Block until every outstanding operation has completed, so
CloseIoRing never runs while the kernel might still be touching a
buffer the ring holds for an in-flight operation.
Waits in short, rechecked steps via SubmitIoRing’s own wait – with
zero new entries queued, its only effect is to block for up to
RUN_DOWN_POLL_MS and reap whatever is already outstanding – rather
than one unbounded call. This does not interpret what it pops: each
popped entry’s payload is dropped, which is sound because its
completion is the proof the kernel has finished with it. Idempotent: calling it
again once outstanding() == 0 is a no-op.
§A poll that expires is not a failure
Each poll blocks for RUN_DOWN_POLL_MS and then reports
ERROR_TIMEOUT if nothing finished in that window, which is the
ordinary outcome for any operation slower than 50ms. Treating that as
an error – which this did until M21.6 – made run_down return Err
with the operation still outstanding, so Drop asserted and then
called CloseIoRing anyway: exactly the “the kernel may still be
writing through a buffer the ring holds” hazard this function exists to
prevent.
This loop therefore has no overall bound, and that is deliberate. Every SQE that successfully queues produces exactly one completion (M10.2), so it terminates. Blocking until that holds is the safe failure mode; closing the ring early is not.
§Choosing an unbounded wait is the caller’s to make
This spelling waits until rundown finishes, however long that takes,
and calling it is how a caller elects that. A caller who wants to
decide for themselves – a deadline, a backoff, a number of attempts
before giving up – calls IoRing::run_down_within instead and owns
the policy entirely. This crate does not implement retry policy
(M26.8): it supplies a bounded primitive and reports what happened.
Note the difference from IoRing::pop_within, which also waits in
segments: there the segments sit inside a period the caller supplied,
which is a bounded wait implemented properly rather than a policy. This
method had segments and no such period, which is what made it the one
waiting API in this crate shaped wrongly.
§Errors
Returns any error from SubmitIoRing other than an expired wait, or
from PopIoRingCompletion. An error leaves the ring resumable: see
IoRing::run_down_within for what is guaranteed about the operations
still queued.
§Panics
As IoRing::try_pop, on a completion for an identity not in flight.
Sourcepub fn run_down_within(&mut self, bound: Duration) -> Result<bool>
pub fn run_down_within(&mut self, bound: Duration) -> Result<bool>
Run down for at most bound, reporting whether it finished.
Ok(true) means nothing is outstanding and the ring is safe to drop.
Ok(false) means the bound elapsed with work still in flight – call
again when your own policy says to. IoRing::outstanding says how
much is left.
§Why this exists, and why it returns rather than retries
Rundown can fail for a reason that a later attempt would survive, and deciding whether to make that attempt is not this crate’s business. A caller running under a deadline, a supervisor with a backoff, and a test that wants to fail fast all want different answers, and a policy baked in here would be wrong for two of the three. So this waits for exactly as long as it is told and then reports.
§What an error guarantees, which is what makes retrying safe
SubmitIoRing documents that “If this function returns an error other
than IORING_E_WAIT_TIMEOUT, then all entries remain in the submission
queue.” So a failure here has not lost the operations and has not
rewound them (D-5); they are still ring state,
a later submit is what runs them, and their buffers must stay alive
until they complete.
The consequence worth stating plainly: after an Err, do not drop
this ring. Dropping it closes a ring the kernel may still write
through, which is the hazard rundown exists to prevent. Call again.
§Errors
Returns any error from SubmitIoRing other than an expired wait, or
from PopIoRingCompletion.
§Panics
As IoRing::try_pop, on a completion for an identity not in flight.
Source§impl<T, X> IoRing<T, X, Open>
impl<T, X> IoRing<T, X, Open>
Sourcepub fn with_inventory(
submission_queue_size: u32,
completion_queue_size: u32,
) -> Result<Self>
pub fn with_inventory( submission_queue_size: u32, completion_queue_size: u32, ) -> Result<Self>
Create a ring whose inventory holds T, negotiating the version as
IoRing::new does.
The generic counterpart to IoRing::new, which exists separately
only so that the common no-payload call keeps inferring its parameter.
§One T per ring
T is a single type, and a pop returns it without a cast because of
that. A caller whose operations carry buffers of different types –
say a win_numa_sys::NumaBuffer arena alongside ad-hoc Vec<u8>
records – therefore cannot put them through one ring as they stand.
Two ways round it:
- A ring per buffer type. Each keeps its own
T, and the operations are independent at the ring level. - An enum payload. One type with a variant per buffer, carrying its
own
unsafe impl IoBuf. The obligation that impl takes on is the same oneIoBufstates, and it has to hold for every variant: the address must not move while the kernel holds it, which for an enum means the bytes must not live inline in the variant.
Erasing T is not among them – see
D-4.
§Errors
As IoRing::new.
Source§impl<T, X, K> IoRing<T, X, K>
impl<T, X, K> IoRing<T, X, K>
Sourcepub fn held(&self) -> usize
pub fn held(&self) -> usize
How many operations this ring is currently holding something for.
Distinct from IoRing::outstanding, which counts every operation the
kernel still owes a completion for. The two are not expected to
be equal: every operation held here is also outstanding, but a _raw
flush or cancel and IoRing::push_raw are outstanding while holding
nothing, by design. So held() <= outstanding() always, and the
difference is the number of raw operations in flight – not a sign of
corruption (D-78). On an Owned ring the
two are equal.
Source§impl<T, X, K: RingKind> IoRing<T, X, K>
impl<T, X, K: RingKind> IoRing<T, X, K>
Sourcepub fn try_pop(&mut self) -> Result<Option<K::Popped<T, X>>>
pub fn try_pop(&mut self) -> Result<Option<K::Popped<T, X>>>
Pop a completion and take back whatever this ring was holding for it.
The replacement for popping a Completion and matching it against a
held token (D-71). The payload is produced by this call and by no
other, which is what makes a use-after-free unrepresentable rather
than merely guarded: there is no way to name an operation and be handed
the memory it may still be using. crate::OperationId deliberately
cannot do it.
§What each None means
There are two, and they answer different questions.
A Some whose payload is None is an operation that never had a
buffer – a flush or a cancellation pushed through an _owned form.
Its sidecar still arrives, which is the point: a flush can say which
group of writes it belonged to.
The outer None means this ring is holding nothing for that
identity because the push that produced it created no entry: a _raw
flush or cancel, or IoRing::push_raw. Those return a bare
user_data and deliberately create no entry – choosing one is
choosing not to have the ring hold anything. Nothing is wrong. M28.5
decided not to give every raw push an entry: that would need an X the
caller never supplied, and the _owned forms already exist for a
caller who wants one.
§On an Owned ring
Neither case of nothing is possible: no call that queues an operation the ring holds
nothing for exists for it, so the return is the flat (completion, payload, sidecar),
with the payload None for an operation that never had a buffer
(D-91).
§Errors
Any error from PopIoRingCompletion.
§Panics
If the completion carries an identity that is not in flight on this
ring – never minted here, already completed, or released after its
build failed. That is a defect, not an outcome
(D-79); during an unwind it is traced
instead, and the completion is returned with an outer None.
Source§impl<T, X> IoRing<T, X, Open>
impl<T, X> IoRing<T, X, Open>
Sourcepub fn into_owned(self) -> Result<IoRing<T, X, Owned>, Self>
pub fn into_owned(self) -> Result<IoRing<T, X, Owned>, Self>
Seal this ring against operations it holds nothing for, so it can be an Owned
ring (D-91).
An Open ring can have in flight an operation the ring holds nothing for: a
registration, a _raw flush or cancel, or IoRing::push_raw. Those are queued
by calls an Owned ring does not have, and the ring this returns can never have one
again; the conversion consumes the ring and there is no way back. So it is refused while
any is in flight – IoRing::outstanding exceeds IoRing::held by exactly that
many – and the ring is handed back, to be popped until it is not.
Nothing is lost: the ring’s identity, its registrations (a RegisteredFiles and a
RegisteredBuffers it made still name it), its completion event and its queue are the
same.
§Errors
The ring, unchanged, if an operation it holds nothing for is in flight.
Source§impl IoRing<()>
Constructors for a ring that holds nothing on a caller’s behalf.
impl IoRing<()>
Constructors for a ring that holds nothing on a caller’s behalf.
These sit on IoRing<()> rather than on the generic impl for a reason that
is about inference, not about capability: a defaulted type parameter
applies in type position, so IoRing::new(..) on a generic impl would be
ambiguous and every existing call site would have to write IoRing::<()>.
A ring that holds payloads is built with
IoRing::with_inventory instead.
Sourcepub fn new(
submission_queue_size: u32,
completion_queue_size: u32,
) -> Result<Self>
pub fn new( submission_queue_size: u32, completion_queue_size: u32, ) -> Result<Self>
Create a ring, negotiating the version as min(RingVersion::HIGHEST_KNOWN, capabilities()?.max_version) (D-6).
§Errors
Returns any error from QueryIoRingCapabilities or CreateIoRing.
Sourcepub fn with_version(
version: RingVersion,
submission_queue_size: u32,
completion_queue_size: u32,
) -> Result<Self>
pub fn with_version( version: RingVersion, submission_queue_size: u32, completion_queue_size: u32, ) -> Result<Self>
Create a ring at exactly version, without negotiation.
§Errors
Returns IoRingError wrapping
IORING_E_VERSION_NOT_SUPPORTED if version exceeds what the system
supports, or any other error from CreateIoRing.
Source§impl<T, X> IoRing<T, X, Open>
impl<T, X> IoRing<T, X, Open>
Sourcepub fn with_version_and_inventory(
version: RingVersion,
submission_queue_size: u32,
completion_queue_size: u32,
) -> Result<Self>
pub fn with_version_and_inventory( version: RingVersion, submission_queue_size: u32, completion_queue_size: u32, ) -> Result<Self>
Source§impl<T, X, K: RingKind> IoRing<T, X, K>
impl<T, X, K: RingKind> IoRing<T, X, K>
Sourcepub fn pop_within(
&mut self,
timeout: Duration,
) -> Result<Option<K::Popped<T, X>>>
pub fn pop_within( &mut self, timeout: Duration, ) -> Result<Option<K::Popped<T, X>>>
Pop one completion, blocking in the ring’s own wait until one is
available or timeout elapses (M21.2).
This is the join between IoRing::try_pop, whose None means
“empty at this instant”, and crate::Batch::submit_and_wait, whose
return deliberately promises nothing about poppability because its
timeout may have expired. Neither one alone answers “give me the
completion I just caused”, and before this existed every caller wrote
that loop again – four different ways across five sites, two of them
unbounded spins.
Ok(None) means the timeout elapsed, or that nothing can arrive:
with no operation outstanding and an empty queue, no completion is
possible, so this returns immediately rather than sleeping out the
full timeout. That early return is what turns “you forgot to submit”
from a timeout into an instant answer.
A zero timeout is exactly one IoRing::try_pop, which is the
honest reading of “wait no time at all”.
Uses SubmitWait, which blocks inside SubmitIoRing with no new
entries queued. It does not touch the ring’s completion event, so
it cannot disturb a caller who owns that event under
D-21. Use
IoRing::pop_within_with to supply a different wait.
§Errors
Any error from SubmitIoRing or PopIoRingCompletion.
§Panics
As IoRing::try_pop, on a completion for an identity not in flight.
Sourcepub fn pop_within_with<W: CompletionWait + ?Sized>(
&mut self,
wait: &mut W,
timeout: Duration,
) -> Result<Option<K::Popped<T, X>>>
pub fn pop_within_with<W: CompletionWait + ?Sized>( &mut self, wait: &mut W, timeout: Duration, ) -> Result<Option<K::Popped<T, X>>>
IoRing::pop_within with a caller-chosen wait.
The crate cannot pick the wait for you, and that is a contract rather than a shrug: the completion event is auto-reset with exactly one waiter per ring (D-21), so a wait this crate chose could consume an edge the caller’s own loop was entitled to. The owner of the ring is the only party who can discharge that obligation, which is why the choice is a parameter.
§Errors
Any error from the wait or from PopIoRingCompletion.
§Panics
As IoRing::try_pop, on a completion for an identity not in flight.