1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83

//! # Introduction
//!
//! A lazy transducer is a declarative specification for the transformation of one type of data to another.
//!
//! The transformer is called the transducer, which receives the original source input, and an index
//! corresponding to the `i`th element, and returns the corresponding `i`th element out of that source.
//!
//! Importantly, lazy transducers are:
//!
//! 1. Lazy - it never parses any elements unless you request it to do so
//! 2. Iterable - one can iterate over every element
//! 3. Indexable - accessing an element is O(1)
//! 4. Parallel - one can iterate in parallel over every element
//!
//! When constructing a lazy transducer, it's important to remember that the transducer must be a `fn`
//! with signature `(Input, usize -> Output)`; a closure which **does not capture its environment** also
//! can be used.
//!
//! The input source needs to be copy; i.e., it should typically be a reference to `Vec<_>`, or a slice of bytes,
//! etc.
//!
//! Another detail that you should keep in mind is that the index is the `i`th element; therefore if
//! your backing data source is bytes, you need to know the _fixed length_ of your data structure you
//! are parsing out of the byte slice. It is up to the transducer implementation to determine this.
//!
//! In some cases, the size of the data structure/output element isn't known statically via `sizeof`, etc.  In these cases
//! it is suggested to pass this "context" in the input source during construction as a tuple, e.g.:
//! `(datasource, context)`, and have your transducer use the `datasource + context + the index` to correctly
//! marshall your data structure out of the bytes.
//!
//! This is the approach that the [scroll-based](type.ScrollTransducer.html) transducer takes.  See also
//! the [bincode example](struct.LazyTransducer.html#advanced-example) for a similar approach.
//!
//! The parallel implementation uses [rayon](https://docs.rs/rayon).
//!
//! # Example
//!
//! ```rust
//! extern crate lazy_transducer;
//! extern crate rayon;
//!
//! use rayon::prelude::*;
//! use lazy_transducer::LazyTransducer;
//!
//! use std::mem::size_of;
//! use std::mem::transmute;
//!
//! # fn main() {
//! let bytes: Vec<u8> = vec![1u8, 0, 2, 0, 3, 0, 4, 0];
//! let lt: LazyTransducer<_, &u16> = LazyTransducer::new(&bytes, 4, |input, index| {
//!     let start = size_of::<u16>() * index;
//!     unsafe { transmute::<_, &u16>(&input[start]) }
//! });
//!
//! // now iterate in parallel
//! lt.into_par_iter().for_each(|n| {
//!   // do stuff
//!   println!("{}", n);
//! });
//! # }
//! ```

mod builder;
pub use crate::builder::*;

mod lazy_transducer;
pub use crate::lazy_transducer::*;

#[cfg(feature = "use_scroll")]
pub use scroll::Endian;

/// The kind of errors for constructing lazy transducers
#[derive(Debug, thiserror::Error)]
pub enum TransducerError {
    #[error("Error during building: {0}")]
    BuilderError(String),
    #[error("Too many elements (size = {nelements:} * {sizeof_element:}) requested from src of size: {src_size:}")]
    ElementOverflow {
        nelements: usize,
        sizeof_element: usize,
        src_size: usize,
    },
}