1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
//! The process logic of protocol layers.
//!
//! This is not a strict OSI stack but rather a group of logical modules to provide a set of
//! intertwined protocols. For example note that the `arp` functionality is also integrated into
//! the `ip` layer as it is required for using the ethernet layer below.
//!
//! ## Layering
//!
//! Each protocol layer is split into two parts; the packet logic contained in `wire` and the
//! processing part in this module. An endpoint represents the local state of a protocol. This
//! state can be used to process packets of that layer. The state is open to modifications as part
//! of a user program while processing does not take place, similar to reconfiguration on the OS
//! level with utilities such as `arp`, `ifconfig`, etc.
//!
//! The general structure of each layer is very similar:
//!
//! * Three types of packet encapsulation: In, Raw, and Out. The first represents an incoming
//! packet with supported features. The second is a packet buffer that can be initialized utilizing
//! the network layers below. And the last is an initialized packet that can be sent outwards.
//!
//! ```text
//! Raw --init-->Out
//! ^ ^|
//! | reinit||into_in
//! | ||
//! \ |v
//! \--deinit--In
//! ```
//!
//! * An endpoint component describing the persistent data of a Host on that layer. A receiver and
//! sender can then make use of the layer by borrowing it while supplying the handler for the
//! next upper layer.
//!
//! ## Receiving
//!
//! Many layer implementations process packets by routing them to layers conceptually above them.
//! This functionality is provided via abstract traits accepting the processed packets of that
//! layer which contain the payload to-be-consumed in the layer above. The encapsulation could be
//! removed if the upper layer does not require any knowledge of the layer below. However, it must
//! be preserved when one wants to use the lower layer for device or protocol specific actions.
//!
//! ## Sending
//!
//! Layers that are capable of send operations (in the sense of packet sending, not logical streams
//! such as TCP) provide a trait that processes Raw packet representations. Initialize the empty
//! packet buffer with data of the layer -- destination address in the case of ip -- and some
//! metadata about the payload that you want to emplace -- most often just the length. Then the Raw
//! packet is converted into an Out packet which offers methods to set the payload while having an
//! initialized and immutable header structure. Finally after inserting the desired payload,
//! request to send the Out packet which will finalize header fields such as checksums and queue
//! the packet buffer in the underlying NIC.
//!
//! ## Answering
//!
//! Many packets require a specified response from a particular layer. With the performance goal in
//! mind but under the constraint of memory allocation we may want to utilize the already valid
//! packet data to construct such an answer in-place of the just received packet. This is important
//! especially for icmp pings and routing functionality. There are two ways to avoid copying the
//! data in that case:
//!
//! * Allocate an additional packet buffer. The extreme of this option allows arbitrary buffer
//! allocation and owning by the user's code which is seldom fit or even possible in resource
//! constrained environments. Since buffers then become a contended resource this creates several
//! DOS risks as well as buffer bloat.
//! * Reinitialize the packet header structures in-place while avoiding to write to any of the
//! payload. In particular each layer calculates the required new length to which the final layer
//! resizes the buffer while ensuring the outer payload is shifted into its new position. Then
//! each layer can emit its representation again into the appropriate place. This is what `ethox`
//! tries to do and should avoid any shifts if the new headers have the same size as the already
//! existing ones.
//!
//! ## In-depth packet representation
//!
//! These are the design goals:
//! * Packet encapsulations may have internal invariants. In particular, the design must not depend
//! on particular implementation of `AsRef<[u8]> + AsMut<[u8]>` to allow this. Thus, the
//! processing pipeline must be able to store a reference to the packet content whose lifetime
//! does not restrict access to other relevant data elsewhere. This needs to be cleanly
//! separated.
//! * Minimize the number of 'callback' arguments, and avoid double dispatch. Single dispatch is
//! okay, and the arguments that it receives should provide all necessary methods to manipulate
//! the content.
//! * Minimize the library magic. As many mechanisms as possible should be open to customization.
//! This includes the protocol receptor implementations but not the core structures of data
//! reprsentations.
//!
//! Only interpreting to a packet's content by referencing the memory region in which it is
//! contained would require reinterpreting all layers on every mutable access at least.
//! Fortunately, there are two classes of types for which we can trust trait implementations
//! sufficiently well: types local to the crate; and types in the standard library. Thus, the
//! actual representation of parsed packet data needs to be separated from the additional data
//! provided by each layer endpoint (which should be implementable by a user as well). The
//! functionality that provides the packet representation is called `Payload` and `PayloadMut`.
//!
//! ## Things that do not work yet – Future work
//!
//! The same instantiation of a layer can not be simply used at multiple points in the callback
//! tree of handlers. Most layer endpoints mutably borrow their persistent state in their send and
//! receive implementations. There are multiple possible ways of avoiding the problem:
//!
//! * Internal mutability and dynamic borrow checking with `RefCell`. The current layers do not
//! need to lend their state to a specific packet. Instead, each method offered by its packet
//! representation mutates some aspects in a local context. The same would also be possible with a
//! shared reference to a `RefCell<_>` of the internal state by calling [`RefCell::borrow_mut`].
//! Note that it would be possible but mildly hazardous to store the so created `RefMut` within
//! the packet given to the upper layer. Such a layer is not re-entrant. If the upper layer is a
//! tunnel of sorts that unpacks an encapsulated lower layer packet then the reentry will fail to
//! again borrow from the `RefCell`.
//! A relevant application of this might be an ip layer used in a wireguard implementation.
//!
//! * Sharding. Split the state into independent fragments, each receiving and sending packets
//! independently. The splitting function can be based on ip subnet or on a hash for example.
//!
//! * Buffer received packets which is definitely the least preferred option.
//!
//! ## Things that do not work – Restrictions
//!
//! ## Previous design choices and Things that need to be thought over
//!
//! TODO: these are somewhat raw thoughts. Expand with justification.
//!
//! How packet buffers are behind a reference, instead of owning a buffer like in some other
//! zerocopy network stacks. There are some points to make either way but Rust's type system gives
//! the former choice a sane interface even when the original pointer is placed within structures.
//! It also works better with the trait design. You can still own the packet buffer but it requires
//! an explicit method on behalf of the specific nic.
//!
//! Receiving not taking an acknowledgement, dropping packets afterwards if they are not used to
//! answer. This also drops some packet buffers where filtered packets could be deinitialized and
//! reused as raw packets immediately.
//!
//! The limited vectorization support and duplicate device handles in nic batched rx/tx. Especially
//! for sending there are overheads in route lookup etc. that could be avoided by batching packet.
//! Might also save on capability information and timestamp queries.
/// A shortened result type for a generic layer operation.
pub type Result<T> = Result;
/// An error type for layer operations.
///
/// These variants explicitely do not capture error cases that happen on the network or as part of
/// the logical protocol interactions but express adverse conditions that are caused by some
/// property or configuration within a layer used during the send or receive process.
/// A standard wrapper for a function implementing receive or send traits.
///
/// Keeps the type alias overhead low by providing a single wrapper type that implements the send
/// and receive traits for all layers, where applicable.
;
/// Can convert from a wire error.
///
/// This indicates some layer tried to operate on a packet but failed.
/// Can convert from a payload error.
///
/// One common cause is failure to resize the buffer to the necessary size.