Trait BlockParser

source

pub trait BlockParser<B: Send + 'static>:
    Clone
    + Send
    + 'static {
    // Required method
    fn extract(&self, block: Block) -> Vec<B>;

    // Provided methods
    fn batch(&self, items: Vec<B>) -> Vec<B> { ... }
    fn options() -> Options { ... }
    fn parse(&self, headers: &[ParsedHeader]) -> Receiver<Result<B>> { ... }
    fn parse_dir(&self, blocks: &str) -> Result<Receiver<Result<B>>> { ... }
    fn parse_ordered(&self, headers: &[ParsedHeader]) -> Receiver<Result<B>> { ... }
    fn parse_with_opts(
        &self,
        headers: &[ParsedHeader],
        opts: Options,
    ) -> Receiver<Result<B>> { ... }
    fn send_batch(&self, tx_c: &Sender<Result<B>>, batch: Result<Vec<B>>) { ... }
}

Expand description

Implement this trait to create a custom Block parser.

Required Methods§

source

fn extract(&self, block: Block) -> Vec

Extracts the data you need from the block.

If you can keep Vec small you will gain memory/speed performance. Always runs on blocks out-of-order using multiple threads so put compute-heavy code in here.

Provided Methods§

source

fn batch(&self, items: Vec) -> Vec

Runs on batches of B to return the final results, blocks will be in-order if Options::order_output has been set.

Implement batch if your algorithm depends on the order of the blocks or if you need to reduce lock contention when accessing shared state in Arc<Mutex<_>>.

Use Options::batch_size if you need to tune the number of the items.

source

fn options() -> Options

The default Options that this parser will use.

Implementing BlockParser::options allows for tuning of the parameters of the algorithm.

source

fn parse(&self, headers: &[ParsedHeader]) -> Receiver<Result>

Parse all the blocks represented by the headers.

source

fn parse_dir(&self, blocks: &str) -> Result<Receiver<Result>>

Parse all the blocks located in the blocks directory

source

fn parse_ordered(&self, headers: &[ParsedHeader]) -> Receiver<Result>

Parse all the blocks represented by the headers, ensuring the results B are returned in the same order the ParsedHeader were passed in.

Note that by ordering the results BlockParser::batch will run on a single thread instead of multiple which could affect performance.

source

fn parse_with_opts( &self, headers: &[ParsedHeader], opts: Options, ) -> Receiver<Result>

Allows users to pass in custom Options in case they need to reduce memory usage or otherwise tune performance for their system. Users should call BlockParser::options to get the default options associated with the parser first.

source