Crate scanlex 

scanlex implements a simple lexical scanner.

Tokens are returned by repeatedly calling the get method, (which will return Token::End if no tokens are left) or by iterating over the scanner.

They represent floats (stored as f64), integers (as i64), characters, identifiers, or single or double quoted strings. There is also Token::Error to indicate a badly formed token. This lexical scanner makes some sensible assumptions, such as a number may not be directly followed by a letter, etc. No attempt is made in this version to decode C-style escape codes in strings. All whitespace is ignored.

Examples

use  scanlex::{Scanner,Token};

let mut scan = Scanner::new("iden 'string' * 10");
assert_eq!(scan.get(),Token::Iden("iden".into()));
assert_eq!(scan.get(),Token::Str("string".into()));
assert_eq!(scan.get(),Token::Char('*'));
assert_eq!(scan.get(),Token::Int(10));
assert_eq!(scan.get(),Token::End);

The scanner struct implements iterator, so:

let v: Vec<_> = scanlex::Scanner::new("bonzo 42 dog (cat)")
    .filter_map(|t| t.to_iden()).collect();
assert_eq!(v,&["bonzo","dog","cat"]);

Structs

ScanError

a scanner error type

ScanLines

used to generate Scanner structs for each line

Scanner

a struct for lexical scanning of a string

Enums

Token

Represents a token returned by Scanner::get