pub trait InputBuffer: Any + Send {
// Required methods
fn flush(&mut self);
fn len(&self) -> BufferSize;
fn hash(&self, hasher: &mut dyn Hasher);
fn take_some(&mut self, n: usize) -> Option<Box<dyn InputBuffer>>;
// Provided methods
fn is_empty(&self) -> bool { ... }
fn take_all(&mut self) -> Option<Box<dyn InputBuffer>> { ... }
}Expand description
A collection of records associated with an input handle.
A Parser holds and adds records to an InputBuffer. The client, which is typically an InputReader, collects one or more InputBuffers and pushes them to the circuit when the controller requests it.
§Pushing buffers into a circuit
There are two ways to push InputBuffers into a circuit:
-
With InputBuffer::flush. This immediately pushes the input buffer into the DBSP input handle.
-
By passing buffers to Parser::stage, which collects all of them into a StagedBuffers. Then, later, call StagedBuffers::flush to push the input buffers into the circuit.
Both approaches are equivalent in terms of correctness. There can be a difference in performance, because InputBuffer::flush has a significant cost for a large number of records. Using StagedBuffers has a similar cost, but it incurs it in the call to Parser::stage rather than in StagedBuffers::flush. This means that, if the input connector can buffer data ahead of the circuit’s demand for it, the cost can be hidden and the circuit as a whole runs faster.
Required Methods§
Sourcefn flush(&mut self)
fn flush(&mut self)
Pushes all of the records into the circuit input handle, and discards those records.
Sourcefn len(&self) -> BufferSize
fn len(&self) -> BufferSize
Returns the number of buffered records and bytes.
Sourcefn hash(&self, hasher: &mut dyn Hasher)
fn hash(&self, hasher: &mut dyn Hasher)
Hashes the records in the input buffer into hasher, in order. This is
used to ensure that input data remains the same for replay, so, for
equal data, it should remain the same from one program run to the next.
Data might be divided into InputBuffers differently from one run to
the next, so the data fed into hasher should be the same if, for
example, records 0..10 then 10..20 are fed in one run and 0..5, 5..15,
15..20 in another.
Sourcefn take_some(&mut self, n: usize) -> Option<Box<dyn InputBuffer>>
fn take_some(&mut self, n: usize) -> Option<Box<dyn InputBuffer>>
Removes the first n records from this input buffer and returns a new
InputBuffer that holds them. If fewer than n records are available,
returns all of them. May return None if this input buffer is empty (or
if n is 0).
Some implementations can’t implement n with full granularity. These
implementations might return more than n records.
This is useful for extracting the records from one of several parser threads to send to a single common thread to be pushed later.
§Byte accounting
This function must not increase or decrease the total number of bytes.
That is, if the returned buffer is named head, self.len().bytes
before the call must equal self.len().bytes + head.len().bytes
following the call. Violating this invariant will cause the number of
buffered bytes reported by a pipeline never to fall to zero (or to wrap
around to u64::MAX).
Provided Methods§
Trait Implementations§
Source§impl InputBuffer for Box<dyn InputBuffer>
impl InputBuffer for Box<dyn InputBuffer>
Source§fn len(&self) -> BufferSize
fn len(&self) -> BufferSize
Source§fn hash(&self, hasher: &mut dyn Hasher)
fn hash(&self, hasher: &mut dyn Hasher)
hasher, in order. This is
used to ensure that input data remains the same for replay, so, for
equal data, it should remain the same from one program run to the next.
Data might be divided into InputBuffers differently from one run to
the next, so the data fed into hasher should be the same if, for
example, records 0..10 then 10..20 are fed in one run and 0..5, 5..15,
15..20 in another.Source§fn flush(&mut self)
fn flush(&mut self)
Source§fn take_some(&mut self, n: usize) -> Option<Box<dyn InputBuffer>>
fn take_some(&mut self, n: usize) -> Option<Box<dyn InputBuffer>>
n records from this input buffer and returns a new
InputBuffer that holds them. If fewer than n records are available,
returns all of them. May return None if this input buffer is empty (or
if n is 0). Read morefn is_empty(&self) -> bool
fn take_all(&mut self) -> Option<Box<dyn InputBuffer>>
Dyn Compatibility§
This trait is dyn compatible.
In older versions of Rust, dyn compatibility was called "object safety".