[−][src]Struct passerine::compiler::lex::Lexer
This represents a lexer object.
A lexer takes a source file and lexes it into tokens.
Note that this struct should not be controlled manually,
use the lex
function instead.
Implementations
impl Lexer
[src]
pub fn new(source: &Rc<Source>) -> Lexer
[src]
Create a new empty lexer.
pub fn all(&mut self) -> Result<Vec<Spanned<Token>>, Syntax>
[src]
Run the lexer, generating the entire token stream.
pub fn step(&self) -> Result<(Token, usize), String>
[src]
Step the lexer, returning the next token.
pub fn remaining(&self) -> &str
[src]
Helper function that returns the remaining source to be lexed as a &str
.
pub fn strip(&mut self)
[src]
Helper function that Strips leading whitespace. Note that a newline is not leading whitespace, it's a separator token.
pub fn expect(source: &str, literal: &str) -> Result<usize, String>
[src]
Helper function that expects an exact literal.
pub fn eat_digits(source: &str) -> Result<usize, String>
[src]
Helper function that eats numeric digits, returning how many lead.
pub fn literal(
source: &str,
literal: &str,
kind: Token
) -> Result<(Token, usize), String>
[src]
source: &str,
literal: &str,
kind: Token
) -> Result<(Token, usize), String>
Helper function that expects a literal, returning an error otherwise.
pub fn open_bracket(source: &str) -> Result<(Token, usize), String>
[src]
Matches a literal opening bracket {
.
pub fn close_bracket(source: &str) -> Result<(Token, usize), String>
[src]
Matches a literal closing bracket `{``.
pub fn unit(source: &str) -> Result<(Token, usize), String>
[src]
Matches a literal closing parenthesis )
.
pub fn open_paren(source: &str) -> Result<(Token, usize), String>
[src]
Matches a literal opening parenthesis (
.
pub fn close_paren(source: &str) -> Result<(Token, usize), String>
[src]
Matches a literal closing parenthesis )
.
pub fn syntax(source: &str) -> Result<(Token, usize), String>
[src]
Matches a macro definition, syntax
.
pub fn assign(source: &str) -> Result<(Token, usize), String>
[src]
Matches a literal assignment equal sign =
.
pub fn lambda(source: &str) -> Result<(Token, usize), String>
[src]
Matches a literal lambda arrow ->
.
pub fn compose(source: &str) -> Result<(Token, usize), String>
[src]
Matches a literal lambda arrow ->
.
pub fn print(source: &str) -> Result<(Token, usize), String>
[src]
Matches a print
expression.
pub fn comment(source: &str) -> usize
[src]
pub fn multi_comment(source: &str) -> usize
[src]
pub fn identifier(source: &str) -> Result<(Token, usize), String>
[src]
Classifies a symbol or a label.
A series of alphanumerics and certain ascii punctuation (see Lexer::is_alpha
).
Can not start with a numeric character.
pub fn symbol(source: &str) -> Result<(Token, usize), String>
[src]
Classifies a symbol (i.e. variable name).
pub fn label(source: &str) -> Result<(Token, usize), String>
[src]
Classifies a label (i.e. data wrapper). Must start with an uppercase character.
pub fn keyword(source: &str) -> Result<(Token, usize), String>
[src]
Classifies a pseudokeyword, used in syntax macros.
Must start with a single quote '
.
pub fn real(source: &str) -> Result<(Token, usize), String>
[src]
Matches a number with a decimal point.
pub fn string(source: &str) -> Result<(Token, usize), String>
[src]
Matches a string, converting escapes.
pub fn boolean(source: &str) -> Result<(Token, usize), String>
[src]
Matches a literal boolean.
pub fn sep(source: &str) -> Result<(Token, usize), String>
[src]
Matches a separator. Note that separators are special, as they're mostly ignored They're used to denote lines in functions blocks. A separator is either a newline or semicolon. They're grouped, so something like ';\n' is only one separator. Although the parser makes no assumptions, there should be only at most one separator between any two non-separator tokens.
Auto Trait Implementations
impl !RefUnwindSafe for Lexer
impl !Send for Lexer
impl !Sync for Lexer
impl Unpin for Lexer
impl UnwindSafe for Lexer
Blanket Implementations
impl<T> Any for T where
T: 'static + ?Sized,
[src]
T: 'static + ?Sized,
impl<T> Borrow<T> for T where
T: ?Sized,
[src]
T: ?Sized,
impl<T> BorrowMut<T> for T where
T: ?Sized,
[src]
T: ?Sized,
pub fn borrow_mut(&mut self) -> &mut T
[src]
impl<T> From<T> for T
[src]
impl<T, U> Into<U> for T where
U: From<T>,
[src]
U: From<T>,
impl<T, U> TryFrom<U> for T where
U: Into<T>,
[src]
U: Into<T>,
type Error = Infallible
The type returned in the event of a conversion error.
pub fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>
[src]
impl<T, U> TryInto<U> for T where
U: TryFrom<T>,
[src]
U: TryFrom<T>,