ibverbs_rs/ibverbs/queue_pair/
mod.rs

1//! Queue pair — the communication endpoint for RDMA operations.
2//!
3//! # The Communication Endpoint
4//!
5//! A [`QueuePair`] is the fundamental object used to send and receive data in RDMA.
6//!
7//! # Key Relationships
8//!
9//! * **Protection Domain (PD)** — A QP is created within a specific [`ProtectionDomain`].
10//!   **Crucial Rule**: You can only use [`MemoryRegion`](crate::ibverbs::memory::MemoryRegion)s
11//!   that were registered in the *same* PD. Mixing PDs will cause immediate errors.
12//!
13//! * **Completion Queues (CQ)** — A QP is associated with two Completion Queues (which can be the same object):
14//!   * **Send CQ** — Receives completions for outgoing operations (Send, Write, Read).
15//!   * **Recv CQ** — Receives completions for incoming operations (Receive).
16//!   * *Note*: When an operation finishes, the hardware places a [`WorkCompletion`](crate::ibverbs::work::WorkCompletion)
17//!     into the corresponding CQ. You must poll that CQ to see the result.
18//!
19//! # Usage: The Post-and-Poll Model
20//!
21//! Using a Queue Pair follows a strict asynchronous pattern:
22//!
23//! 1.  **Post**: You submit a work request using methods like [`post_send`](QueuePair::post_send).
24//!     This method returns immediately (non-blocking).
25//! 2.  **Execute**: The hardware processes the request asynchronously in the background.
26//! 3.  **Complete**: Eventually, a completion event appears in the associated CQ. You poll the CQ
27//!     using the `wr_id` you assigned to retrieve the result.
28//!
29//! # Safety Model
30//!
31//! The `post_*` methods are **`unsafe`** because they do not enforce buffer lifetime safety automatically.
32//!
33//! ## The Contract
34//!
35//! When you create a [`WorkRequest`](crate::ibverbs::work), the [`GatherElement`](crate::ibverbs::memory::GatherElement)
36//! and [`ScatterElement`](crate::ibverbs::memory::ScatterElement) types capture the lifetime of your data buffers
37//! through Rust's borrow checker. However, **the `QueuePair` does not return a handle** that ties this
38//! lifetime to the completion of the operation.
39//!
40//! **It is your responsibility to**:
41//! 1.  Keep the data buffers alive until the operation completes.
42//! 2.  Not mutate (for Send/Write) or access (for Receive/Read) the buffers while the hardware owns them.
43//! 3.  Poll the appropriate Completion Queue using the `wr_id` to know when the operation finishes.
44//!
45//! Only after you receive the [`WorkCompletion`](crate::ibverbs::work::WorkCompletion) is it safe to
46//! alias, drop, reuse, or modify the buffers.
47//!
48//! ## Safe Abstractions
49//!
50//! A higher-level abstraction [`Channel`](crate::channel::Channel) wraps these `unsafe` methods and enforces
51//! the lifetime contract by returning a future or handle that must be awaited/polled before the
52//! buffers are released.
53
54pub mod builder;
55pub mod config;
56pub mod ops;
57
58use crate::ibverbs::completion_queue::CompletionQueue;
59use crate::ibverbs::error::IbvError;
60use crate::ibverbs::protection_domain::ProtectionDomain;
61use ibverbs_sys::{ibv_destroy_qp, ibv_qp};
62use std::fmt::Debug;
63
64/// An RDMA Queue Pair.
65///
66/// This struct manages the lifecycle of the QP resource. It holds strong references to its
67/// dependencies ([`ProtectionDomain`] and [`CompletionQueue`]) to ensure they remain
68/// allocated as long as this QP exists.
69#[doc(alias = "ibv_qp")]
70#[doc(alias = "ibv_create_qp")]
71pub struct QueuePair {
72    pd: ProtectionDomain,
73    // Kept to ensure CQs are not dropped before the QP
74    _send_cq: CompletionQueue,
75    _recv_cq: CompletionQueue,
76    qp: *mut ibv_qp,
77}
78
79/// SAFETY: libibverbs resources are thread-safe.
80unsafe impl Send for QueuePair {}
81/// SAFETY: libibverbs resources are thread-safe.
82unsafe impl Sync for QueuePair {}
83
84impl Drop for QueuePair {
85    fn drop(&mut self) {
86        log::debug!("QueuePair destroyed");
87        let errno = unsafe { ibv_destroy_qp(self.qp) };
88        if errno != 0 {
89            let error = IbvError::from_errno_with_msg(errno, "Failed to destroy queue pair");
90            log::error!("{error}");
91        }
92    }
93}
94
95impl Debug for QueuePair {
96    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> Result<(), std::fmt::Error> {
97        f.debug_struct("QueuePair")
98            .field("handle", &unsafe { (*self.qp).handle })
99            .field("qp_num", &unsafe { (*self.qp).qp_num })
100            .field("state", &unsafe { (*self.qp).state })
101            .field("type", &unsafe { (*self.qp).qp_type })
102            .field("send_cq_handle", &unsafe { (*(*self.qp).send_cq).handle })
103            .field("recv_cq_handle", &unsafe { (*(*self.qp).recv_cq).handle })
104            .field("pd", &self.pd)
105            .finish()
106    }
107}
ibverbs_rs/ibverbs/queue_pair/mod.rs

ibverbs_rs/ibverbs/queue_pair/
mod.rs
