Struct Scanner 

pub struct Scanner<'a> { /* private fields */ }

A lexer to split a string into a list of Tokens.

Implementations

impl<'a> Scanner<'a>

pub fn tokenize(source: &'a str) -> Result<Vec<Token>>

Converts an input string into a list of Tokens.

Examples

use slac::{Scanner, Token, Value};

let tokens = Scanner::tokenize("40 + 2").unwrap();
let expected: Vec<Token> = vec![Token::Literal(Value::Number(40.0)), Token::Plus, Token::Literal(Value::Number(2.0))];

assert_eq!(tokens, expected);

Errors

Returns an Error when encountering invalid input.