Struct tokenizer_lib::ParallelTokenQueue

pub struct ParallelTokenQueue;

A token queue used for doing lexing and parsing on different threads. Will send tokens between threads

Implementations

impl ParallelTokenQueue

pub fn new<T: PartialEq, TData>(
) -> (ParallelTokenSender<T, TData>, ParallelTokenReader<T, TData>)

Creates two items, a sender and a receiver. Where the reader is on the parsing thread and the sender is on the lexer thread

pub fn new_with_buffer_size<T: PartialEq, TData>(
    buffer_size: usize
) -> (ParallelTokenSender<T, TData>, ParallelTokenReader<T, TData>)