Struct LazyTransducer

pub struct LazyTransducer<'a, Input, Output>
where
    Input: 'a + Copy,
    Output: 'a,
{ /* private fields */ }

A lazy transducer transforms n elements from a source type into an output type.

The transformer is called the transducer, which receives the original source input, and an index corresponding to the ith element, and returns the corresponding ith element out of that source.

Importantly, lazy transducers are:

1. Lazy - it never parses any elements unless you request it to do so
2. Iterable - one can iterate over every element
3. Indexable - accessing an element is O(1)
4. Parallel - one can iterate in parallel over every element

Basic Example

We need to provide a data source, the number of elements in the data source, and the means of extracting the elements out of the data source (the transducer).

For a simple case, we can consider a backing array of u32s, which we cast to u64s.

extern crate lazy_transducer;
use lazy_transducer::LazyTransducer;

let data = [0xdeadbeefu32, 0xcafed00d];
let lt: LazyTransducer<&[u32], u64> = LazyTransducer::new(&data, 2, |input, idx| input[idx] as u64);

let cafedood = lt.get(1).expect("has 2 elements");
assert_eq!(cafedood, 0xcafed00d);

for (i, elem) in lt.into_iter().enumerate() {
  println!("{}: {}", i, elem);
}

Advanced Example

This example uses the bincode binary serializer as its transducer.

extern crate lazy_transducer;
#[macro_use]
extern crate serde_derive;
extern crate serde;
extern crate bincode;
extern crate rayon;

use lazy_transducer::LazyTransducer;
use bincode::{serialize, deserialize, Error};
use rayon::prelude::*;

#[derive(Debug, PartialEq, Serialize, Deserialize)]
pub struct Foo {
  x: u64,
  y: f32,
  z: bool,
}

fn run() -> Result<(), Error> {
  let foo1 = Foo { x: 0xcafed00d, y: 0.75, z: false };
  let foo2 = Foo { x: 0xdeadbeef, y: 0.50, z: true };

  // we need to serialize the data, which we do by extending a byte vector with the individually
  // serialized components
  let mut data = serialize(&foo1)?;
  let sizeof_serialized_element = data.len();
  data.extend_from_slice(&serialize(&foo2)?);

  // we construct our transducer by providing the serialized bytes _and_ the size of a serialized
  // element as input; our transducer just reads at the appropriate byte offset, and deserializes!
  let lt: LazyTransducer<_, Result<Foo, Error>> =
    LazyTransducer::new((data.as_slice(), sizeof_serialized_element),
                        2,
                        |(input, size), idx| {
                           deserialize(&input[(idx * size)..])
  });

  let foo2_ = lt.get(1).expect("has 2 elements")?;
  assert_eq!(foo2, foo2_);

  // and now with the help of rayon, we iterate over the items in parallel
  lt.into_par_iter().for_each(|elem| {
    println!("{:?}", elem);
  });
  Ok(())
}

Implementations

impl<'a, Input, Output> LazyTransducer<'a, Input, Output>

where Input: 'a + Copy, Output: 'a,

pub fn len(&self) -> usize

How many elements are contained in this lazy transducer

pub fn new( contents: Input, count: usize, transducer: fn(Input, usize) -> Output, ) -> Self

Create a new LazyTransducer with count elements in contents, using transducer to extract them.

Example

use lazy_transducer::LazyTransducer;
let data = [0xdeadbeefu32, 0xcafed00d];
let lt: LazyTransducer<&[u32], u64> = LazyTransducer::new(&data, 2, |input, idx| input[idx] as u64);

pub fn get(&self, idx: usize) -> Option<Output>

Get an element out of the lazy transducer, returning None if the index is greater than the number of elements in this lazy transducer.

Example

extern crate lazy_transducer;
use lazy_transducer::LazyTransducer;

let data = [0xdeadbeefu64, 0xcafed00d];
let lt: LazyTransducer<&[u64], &u64> = LazyTransducer::new(&data, 2, |input, idx| &input[idx]);

let cafedood = lt.get(1).expect("has 2 elements");
assert_eq!(*cafedood, 0xcafed00d);

assert!(lt.get(2).is_none());

Trait Implementations

impl<'a, Input: Copy, Output> Clone for LazyTransducer<'a, Input, Output>

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<'a, Input, Output> Debug for LazyTransducer<'a, Input, Output>

where Input: 'a + Copy + Debug, Output: 'a + Debug,

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'a, 'b, Input: Copy, Output> IntoIterator for &'b LazyTransducer<'a, Input, Output>

type Item = Output

The type of the elements being iterated over.

type IntoIter = IntoIter<'a, Input, Output>

Which kind of iterator are we turning this into?

fn into_iter(self) -> Self::IntoIter

Creates an iterator from a value.

impl<'a, Input: Copy, Output> IntoIterator for LazyTransducer<'a, Input, Output>

type Item = Output

The type of the elements being iterated over.

type IntoIter = IntoIter<'a, Input, Output>

Which kind of iterator are we turning this into?

fn into_iter(self) -> Self::IntoIter

Creates an iterator from a value.

impl<'a, Input: Sync + Copy + Send, Output: Send + Sync> IntoParallelIterator for LazyTransducer<'a, Input, Output>

type Iter = IntoParIter<'a, Input, Output>

The parallel iterator type that will be created.

type Item = Output

The type of item that the parallel iterator will produce.

fn into_par_iter(self) -> Self::Iter

Converts self into a parallel iterator.