Struct differential_dataflow::trace::implementations::spine_fueled_neu::Spine

pub struct Spine<K, V, T: Lattice + Ord, R: Semigroup, B: Batch<K, V, T, R>> { /* fields omitted */ }

An append-only collection of update tuples.

A spine maintains a small number of immutable collections of update tuples, merging the collections when two have similar sizes. In this way, it allows the addition of more tuples, which may then be merged with other immutable collections.

Methods

impl<K, V, T, R, B> Spine<K, V, T, R, B> where
    K: Ord + Clone,
    V: Ord + Clone,
    T: Lattice + Ord + Clone + Debug + Default,
    R: Semigroup,
    B: Batch<K, V, T, R>, 

pub fn with_effort(
    effort: usize,
    operator: OperatorInfo,
    logger: Option<Logger>,
    activator: Option<Activator>
) -> Self

Allocates a fueled Spine with a specified effort multiplier.

This trace will merge batches progressively, with each inserted batch applying a multiple of the batch's length in effort to each merge. The effort parameter is that multiplier. This value should be at least one for the merging to happen; a value of zero is not helpful.

pub fn introduce_batch(&mut self, batch: Option<B>, batch_index: usize)

Introduces a batch at an indicated level.

The level indication is often related to the size of the batch, but it can also be used to artificially fuel the computation by supplying empty batches at non-trivial indices, to move merges along.

pub fn apply_fuel(&mut self, fuel: &mut isize)

Applies an amount of fuel to merges in progress.

The supplied fuel is for each in progress merge, and if we want to spend the fuel non-uniformly (e.g. prioritizing merges at low layers) we could do so in order to maintain fewer batches on average (at the risk of completing merges of large batches later, but tbh probably not much later).

Trait Implementations

impl<K, V, T, R, B> TraceReader for Spine<K, V, T, R, B> where
    K: Ord + Clone,
    V: Ord + Clone,
    T: Lattice + Ord + Clone + Debug + Default,
    R: Semigroup,
    B: Batch<K, V, T, R> + Clone + 'static, 

type Key = K

Key by which updates are indexed.

type Val = V

Values associated with keys.

type Time = T

Timestamps associated with updates

type R = R

Associated update.

type Batch = B

The type of an immutable collection of updates.

type Cursor = CursorList<K, V, T, R, <B as BatchReader<K, V, T, R>>::Cursor>

The type used to enumerate the collections contents.

fn cursor_through(
    &mut self,
    upper: &[T]
) -> Option<(Self::Cursor, <Self::Cursor as Cursor<K, V, T, R>>::Storage)>

Acquires a cursor to the restriction of the collection's contents to updates at times not greater or equal to an element of upper.

fn advance_by(&mut self, frontier: &[T])

Advances the frontier of times the collection must be correctly accumulable through.

fn advance_frontier(&mut self) -> &[T]

Reports the frontier from which all time comparisions should be accurate.

fn distinguish_since(&mut self, frontier: &[T])

Advances the frontier that may be used in cursor_through.

fn distinguish_frontier(&mut self) -> &[T]

Reports the frontier from which the collection may be subsetted.

fn map_batches<F: FnMut(&Self::Batch)>(&mut self, f: F)

Maps logic across the non-empty sequence of batches in the trace.

fn cursor(
    &mut self
) -> (Self::Cursor, <Self::Cursor as Cursor<Self::Key, Self::Val, Self::Time, Self::R>>::Storage)

Provides a cursor over updates contained in the trace.

fn read_upper(&mut self, target: &mut Antichain<Self::Time>) where
    Self::Time: Timestamp, 

Reads the upper frontier of committed times.

fn advance_upper(&mut self, upper: &mut Antichain<Self::Time>) where
    Self::Time: Timestamp, 

Advances upper by any empty batches.

impl<K, V, T, R, B> Trace for Spine<K, V, T, R, B> where
    K: Ord + Clone,
    V: Ord + Clone,
    T: Lattice + Ord + Clone + Debug + Default,
    R: Semigroup,
    B: Batch<K, V, T, R> + Clone + 'static, 

fn new(
    info: OperatorInfo,
    logging: Option<Logger>,
    activator: Option<Activator>
) -> Self

Allocates a new empty trace.

fn exert(&mut self, effort: &mut isize)

Apply some amount of effort to trace maintenance.

The units of effort are updates, and the method should be thought of as analogous to inserting as many empty updates, where the trace is permitted to perform proportionate work.

fn insert(&mut self, batch: Self::Batch)

Introduces a batch of updates to the trace.

fn close(&mut self)

Completes the trace with a final empty batch.