Module parser

Syntax parser for the shell language

The shell language parsing system has two important components: the lexical analyzer and the syntax parser.

The lexical analyzer, or simply lexer, extracts tokens from the source code string. Tokenization for the shell language involves parsing expansions like parameter expansions and command substitutions, which makes the lexer much more complicated than those for normal programming languages. However, as long as you use the lexer indirectly via the parser, you don’t have to care about such details.

The syntax parser examines tokens produced by the lexer and constructs abstract syntax trees. The below code illustrates the basic usage of the parser.

// First, prepare an input object that the lexer reads from.
use yash_syntax::input::Memory;
let input = Box::new(Memory::new("echo $?"));

// Next, create a lexer.
use yash_syntax::parser::lex::Lexer;
let mut lexer = Lexer::new(input);

// Then, create a new parser borrowing the lexer.
use yash_syntax::parser::Parser;
let mut parser = Parser::new(&mut lexer);

// Lastly, call the parser's function to get an AST.
use futures_executor::block_on;
let list = block_on(parser.command_line()).unwrap().unwrap();
assert_eq!(list.to_string(), "echo $?");

If there is any error reading the input or analyzing the source code, the parser returns an Error object. In case of a syntax error, the Error object’s cause will be a value of SyntaxError that describes it.

Most lexer and parser functions are asynchronous because underlying input is asynchronous. Only as many lines are read from the input as needed to parse a complete AST.

Note that most AST types have the FromStr trait implemented for them. If you don’t need to include source location information in the resultant AST, calling the parse function on a string is a convenient way to parse a code fragment. See the syntax module for an example of this.

Modules

lex

Lexical analyzer

Structs

Config

Set of parameters for constructing a parser

Error

Explanation of a failure in parsing

Parser

The shell syntax parser

Enums

ErrorCause

Types of errors that may happen in parsing

Rec

Modifier that makes a result of parsing optional in order to trigger the parser to restart parsing after alias substitution

SyntaxError

Types of syntax errors

Type Aliases

Result

Entire result of parsing