1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
#[cfg(doc)]
use {crate::iterators::ConsIter, crate::iterators::Detached, core::mem::MaybeUninit};
use crate::iterators::iterator_trait::{ORBIterator, PrivateORBIterator};
use crate::iterators::sync_iterators::Iter;
use crate::iterators::{copy_from_slice_unchecked, private_impl};
use crate::ring_buffer::iters_components::PIterComponent;
use crate::ring_buffer::storage_components::{PStorageComponent, StorageComponent};
use crate::ring_buffer::wrappers::refs::IntoRef;
use crate::ring_buffer::wrappers::unsafe_sync_cell::UnsafeSyncCell;
use crate::ring_buffer::{OneRB, SharedRB};
#[cfg(feature = "async")]
use crate::{
iterators::{AsyncProdIter, async_iterators::AsyncIterator},
iters_components::async_iters::AsyncIterComp,
};
#[doc = r##"
Iterator used to push data into the buffer.
When working with types which implement both
[`Copy`](https://doc.rust-lang.org/std/marker/trait.Copy.html) and
[`Clone`](https://doc.rust-lang.org/std/clone/trait.Clone.html) traits, `copy` methods should be
preferred over `clone` methods.
# TL;DR about uninitialised memory
If you created the buffer with either `default` or `from` methods and you are *not* going to use [`ConsIter::pop`],
then it is safe to use normal methods from this struct.
If you either created the buffer with `new_zeroed` or are going to move items out of the buffer (e.g. using [`ConsIter::pop`]),
then you must pay attention to what you do and ensure that all the locations cleared by `pop` will be re-filled with `*_init` methods.
After that you can use normal methods again.
Read below to know why and how.
It would be a good idea to do a check with [miri](https://github.com/rust-lang/miri), which is able to
tell if and when something bad has happened.
# A note about how this buffer is made:
Every element in this buffer is wrapped in an [`UnsafeSyncCell`], which in the end is a [`MaybeUninit`].
`MaybeUninit` (read the official docs if you want to know more) is a way to deal with possibly uninitialised data.
When one creates a buffer from this crate, they may choose to build it either with `default` and `from` methods,
or with `new_zeroed` ones.
When using the former, an initialised buffer is created, so there are no problems concerning uninitialised memory.
With the latter methods, a zeroed buffer is rather created.
To write data into an uninitialised (or zeroed) block of memory, one has to use [`write`](https://doc.rust-lang.org/std/primitive.pointer.html#method.write)
method, which overwrites a memory location, without reading or dropping the old value.
Remember that a zeroed location must *never* be read or dropped. Doing so would cause UB!
On the other hand, using `write` on an initialised block doesn't drop the old value, causing then a memory leak.
For each of the methods in this struct, there exists a `*_init` one (e.g. [`Self::push`] and [`Self::push_init`]).
Normal methods are faster than `*_init` ones, and should be preferred over these when dealing with *surely*
initialised memory.
On the other hand, `*_init methods` always perform a check over the memory they are going to write and choose the proper way to
deal it, even dropping the old value, if there is the need. So they are safe to use upon a possibly uninitialised block.
"##]
#[repr(transparent)]
pub struct ProdIter<B: IntoRef + OneRB> {
inner: Iter<B>,
}
unsafe impl<B: IntoRef + OneRB + SharedRB> Send for ProdIter<B> {}
impl<B: IntoRef + OneRB<Item = T>, T> PrivateORBIterator for ProdIter<B> {
type _Buffer = B;
#[inline]
fn _available(&mut self) -> usize {
let succ_idx = self.succ_index();
unsafe {
self.inner.cached_avail = match self.inner.index < succ_idx {
true => succ_idx.unchecked_sub(self.inner.index).unchecked_sub(1),
false => self
.inner
.buffer
.storage()
.len()
.unchecked_sub(self.inner.index)
.unchecked_add(succ_idx)
.unchecked_sub(1),
};
}
self.inner.cached_avail
}
#[inline]
fn set_atomic_index(&self, index: usize) {
self.inner.buffer.iters().set_prod_index(index);
}
#[inline]
fn succ_index(&self) -> usize {
self.inner.buffer.iters().cons_index()
}
private_impl!();
}
impl<B: IntoRef + OneRB<Item = T>, T> ORBIterator for ProdIter<B> {
type Item = T;
type Buffer = B;
}
#[cfg(feature = "async")]
impl<B: IntoRef + OneRB<Iters: AsyncIterComp>> ProdIter<B> {
pub fn into_async(self) -> AsyncProdIter<B> {
AsyncIterator::from_sync(self)
}
}
impl<B: IntoRef + OneRB<Item = T>, T> ProdIter<B> {
pub(crate) fn new(value: B::TargetRef) -> Self {
Self {
inner: Iter::new(value),
}
}
#[inline]
fn _push(&mut self, value: T, f: fn(*mut T, T)) -> Result<(), T> {
if let Some(binding) = self.next_ref_mut_init() {
f(binding, value);
unsafe { self.advance(1) };
Ok(())
} else {
Err(value)
}
}
/// Tries to push a new item by moving or copying it.
///
/// This method must *not* be used to push items after a [`ConsIter::pop`].
/// In this case, [`Self::push_init`] has to be used, instead.
/// For more info, refer to the main documentation above.
///
/// Returns:
/// * `Err(value)`, if the buffer is full;
/// * `Ok(())`, otherwise.
#[inline]
pub fn push(&mut self, value: T) -> Result<(), T> {
fn f<T>(binding: *mut T, value: T) {
unsafe {
*binding = value;
}
}
self._push(value, f)
}
/// Same as [`Self::push_slice`], but can be used when dealing with possibly uninitialised
/// locations within the buffer, e.g. after a [`ConsIter::pop`].
///
/// Returns:
/// * `Err(value)`, if the buffer is full;
/// * `Ok(())`, otherwise.
#[inline]
pub fn push_init(&mut self, value: T) -> Result<(), T> {
fn f<T>(binding: *mut T, value: T) {
unsafe {
if UnsafeSyncCell::check_zeroed(binding) {
binding.write(value);
} else {
*binding = value;
}
}
}
self._push(value, f)
}
#[inline]
fn _push_slice(&mut self, slice: &[T], f: fn(&mut [T], &[T])) -> Option<()> {
let count = slice.len();
self.check(count).then(|| {
self.inner
.buffer
.storage_mut()
._push_slice(self.inner.index, slice, f);
unsafe { self.advance(count) };
})
}
/// Tries to push a slice of items by copying the elements.
/// The elements must implement [`Copy`](https://doc.rust-lang.org/std/marker/trait.Copy.html) trait.
///
/// This method must *not* be used to push items after a [`ConsIter::pop`].
/// In this case, [`Self::push_slice_init`] has to be used, instead.
/// For more info, refer to the main documentation above.
///
/// Returns:
/// * `None`, if the buffer is full;
/// * `Some(())`, otherwise.
#[inline]
pub fn push_slice(&mut self, slice: &[T]) -> Option<()>
where
T: Copy,
{
#[inline]
fn f<T: Copy>(binding: &mut [T], slice: &[T]) {
copy_from_slice_unchecked(slice, binding);
}
self._push_slice(slice, f)
}
/// Same as [`Self::push_slice`], but can be used when dealing with possibly uninitialised
/// locations within the buffer, e.g. after a [`ConsIter::pop`].
///
/// Returns:
/// * `None`, if the buffer is full;
/// * `Some(())`, otherwise.
#[inline]
pub fn push_slice_init(&mut self, slice: &[T]) -> Option<()>
where
T: Copy,
{
#[inline]
fn f<T: Copy>(binding_h: &mut [T], slice: &[T]) {
for (x, y) in binding_h.iter_mut().zip(slice) {
if UnsafeSyncCell::check_zeroed(x as *mut T) {
unsafe {
(x as *mut T).write(*y);
}
} else {
*x = *y;
}
}
}
self._push_slice(slice, f)
}
/// Tries to push a slice of items by cloning the elements.
/// The elements must implement [`Clone`](https://doc.rust-lang.org/std/clone/trait.Clone.html) trait.
///
/// This method must *not* be used to push items after a [`ConsIter::pop`].
/// In this case, [`Self::push_slice_clone_init`] has to be used, instead.
/// For more info, refer to the main documentation above.
///
/// Returns:
/// * `None`, if the buffer is full;
/// * `Some(())`, otherwise.
#[inline]
pub fn push_slice_clone(&mut self, slice: &[T]) -> Option<()>
where
T: Clone,
{
#[inline]
fn f<T: Clone>(binding_h: &mut [T], slice: &[T]) {
binding_h.clone_from_slice(slice);
}
self._push_slice(slice, f)
}
/// Same as [`Self::push_slice_clone`], but can be used when dealing with possibly uninitialised
/// locations within the buffer, e.g. after a [`ConsIter::pop`].
///
/// Returns:
/// * `None`, if the buffer is full;
/// * `Some(())`, otherwise.
#[inline]
pub fn push_slice_clone_init(&mut self, slice: &[T]) -> Option<()>
where
T: Clone,
{
#[inline]
fn f<T: Clone>(binding_h: &mut [T], slice: &[T]) {
for (x, y) in binding_h.iter_mut().zip(slice) {
if UnsafeSyncCell::check_zeroed(x as *mut T) {
unsafe {
(x as *mut T).write(y.clone());
}
} else {
x.clone_from(y);
}
}
}
self._push_slice(slice, f)
}
/// If available, returns a mutable reference to the next item.
/// This reference can be used to write data into an *initialised* item.
///
/// Items can be initialised by calling [`Self::get_next_item_mut_init`] or by creating a buffer
/// using `default` constructor. E.g.: `SharedHeapRB::default` or `LocalStackRB::default`.
///
/// For uninitialised items, use [`Self::get_next_item_mut_init`], instead.
///
/// <div class="warning">
///
/// Being this a reference, [`Self::advance`] has to be called when done with the mutation
/// in order to move the iterator.
/// </div>
///
/// # Safety
/// The retrieved item must be initialised! For more info, refer to [`MaybeUninit::assume_init_mut`](https://doc.rust-lang.org/std/mem/union.MaybeUninit.html#method.assume_init_mut).
pub unsafe fn get_next_item_mut<'a>(&mut self) -> Option<&'a mut T> {
self.next_ref_mut()
}
/// If available, returns a mutable pointer to the next item.
/// This pointer can be used to write data into the item, even if this is not already initialised.
/// It is important to note that reading from this pointer or turning it into a reference is still
/// undefined behavior, unless the item is initialised.
///
/// If the memory pointed by this pointer is already initialised, it is possible to write into it
/// with a simple:
/// ```ignore
/// *ptr = value;
/// ```
/// Doing so, the old value will be automatically dropped and no leak will be created.
///
/// If the memory is not initialised, the write must be done with:
/// ```ignore
/// ptr.write(value);
/// ```
/// The reason is that `write` does not drop the old value, which is good, because dropping an
/// uninitialised value is UB!
///
/// One should be able to test whether a piece of memory is initialised with [`UnsafeSyncCell::check_zeroed`].
///
/// For more info, refer to [`MaybeUninit::as_mut_ptr`](https://doc.rust-lang.org/std/mem/union.MaybeUninit.html#method.as_mut_ptr).
/// <div class="warning">
///
/// Being this a pointer, [`Self::advance`] has to be called when done with the mutation
/// in order to move the iterator.
/// </div>
pub fn get_next_item_mut_init(&mut self) -> Option<*mut T> {
self.next_ref_mut_init()
}
/// If available, returns two mutable slices with a total count equal to `count`.
/// These references can be used to write data into *initialised* items.
///
/// Items can be initialised (one by one) by calling [`Self::get_next_item_mut_init`] or by creating a buffer
/// using `default` constructor. E.g.: `SharedHeapRB::default` or `LocalStackRB::default`.
///
/// <div class="warning">
///
/// Being these reference, [`Self::advance`] has to be called when done with the mutation
/// in order to move the iterator.
/// </div>
///
/// # Safety
/// The retrieved items must be initialised! For more info, refer to [`MaybeUninit::assume_init_mut`](https://doc.rust-lang.org/std/mem/union.MaybeUninit.html#method.assume_init_mut).
pub unsafe fn get_next_slices_mut<'a>(
&mut self,
count: usize,
) -> Option<<<B as OneRB>::Storage as StorageComponent>::SliceOutputMut<'a>> {
use crate::ring_buffer::storage_components::PStorageComponent;
self.check(count).then(|| {
self.inner
.buffer
.storage_mut()
.next_chunk_mut(self.inner.index, count)
})
}
}
pub mod test {
#[test]
fn cached_avail() {
use super::*;
const BUFFER_SIZE: usize = 4095;
#[cfg(feature = "alloc")]
let buf = crate::SharedHeapRB::<u32>::default(BUFFER_SIZE + 1);
#[cfg(not(feature = "alloc"))]
let mut buf = crate::SharedStackRB::<u32, { BUFFER_SIZE + 1 }>::default();
let (mut prod, mut cons) = buf.split();
assert_eq!(prod.inner.cached_avail, 0);
prod.check(1);
assert_eq!(prod.inner.cached_avail, BUFFER_SIZE);
unsafe {
prod.advance(2);
}
assert_eq!(prod.inner.cached_avail, BUFFER_SIZE - 2);
unsafe {
cons.advance(1);
}
assert_eq!(prod.inner.cached_avail, BUFFER_SIZE - 2);
prod.check(10);
assert_eq!(prod.inner.cached_avail, BUFFER_SIZE - 2);
}
}