Trait sqlite3_parser::lexer::Splitter

pub trait Splitter: Sized {
    type Error: ScanError;
    type TokenType;
    fn split<'input>(
        &mut self, 
        data: &'input [u8], 
        eof: bool
    ) -> Result<(Option<(&'input [u8], Self::TokenType)>, usize), Self::Error>;
}

Split function used to tokenize the input

Associated Types

type Error: ScanError

type TokenType

Loading content...

Required methods

fn split<'input>(
    &mut self,
    data: &'input [u8],
    eof: bool
) -> Result<(Option<(&'input [u8], Self::TokenType)>, usize), Self::Error>

The arguments are an initial substring of the remaining unprocessed data and a flag, eof, that reports whether the Reader has no more data to give.

If the returned error is non-nil, scanning stops and the error is returned to the client.

The function is never called with an empty data slice unless at EOF. If eof is true, however, data may be non-empty and, as always, holds unprocessed text.

Loading content...

Implementors

impl Splitter for Tokenizer

type Error = Error

type TokenType = TokenType

fn split<'input>(
    &mut self,
    data: &'input [u8],
    eof: bool
) -> Result<(Option<Token<'input>>, usize), Error>

Loading content...