Struct Parser

pub struct Parser<'a, 'b> { /* private fields */ }

The shell syntax parser

A parser manages a set of data used in syntax parsing. It keeps a reference to a lexer that provides tokens to parse. It also has some parameters that can be set by a configuration and affect the parsing process.

The new function directly creates a parser with default settings. If you want to customize the settings, you can use the config function to create a configuration and then create a parser with the configuration.

Parsing here-documents

Most intrinsic functions of Parser may return an AST containing HereDocs with empty content. The parser creates the HereDoc instance when it finds a here-document operator, but it has not read its content at that time. When finding a newline token, the parser reads the content and fills it into the HereDoc instance.

Unless you are interested in parsing a specific syntactic construct that is only part of source code, you will want to use a function that returns a complete result filled with proper here-document contents if any. Then the command_line function is for you. See also the module documentation.

Implementations

impl<'a, 'b> Parser<'a, 'b>

pub fn config() -> Config<'a>

Creates a new configuration with default settings.

This is a synonym for Config::new. Customize the settings by calling methods of the returned configuration and then create a parser by calling its input method.

pub fn new(lexer: &'a mut Lexer<'b>) -> Parser<'a, 'b>

Creates a new parser based on the given lexer.

The parser uses the lexer to read tokens. All other settings are default. To customize the settings, use the config function.

pub async fn peek_token(&mut self) -> Result<&Token>

Returns a reference to the current token.

If the current token is not yet read from the underlying lexer, it is read.

pub async fn take_token_raw(&mut self) -> Result<Token>

Consumes the current token without performing alias substitution.

If the current token is not yet read from the underlying lexer, it is read.

This function does not perform alias substitution and therefore should be used only in context where no alias substitution is expected. Otherwise, you should use take_token_manual or take_token_auto instead.

pub async fn take_token_manual( &mut self, is_command_name: bool, ) -> Result<Rec<Token>>

Consumes the current token after performing applicable alias substitution.

If the current token is not yet read from the underlying lexer, it is read.

This function checks if the token is the name of an alias. If it is, alias substitution is performed on the token and the result is Ok(AliasSubstituted). Otherwise, the token is consumed and returned.

Alias substitution is performed only if at least one of the following is true:

• The token is the first command word in a simple command, that is, it is the word for the command name. (This condition should be specified by the is_command_name parameter.)
• The token comes just after the replacement string of another alias substitution that ends with a blank character.
• The token names a global alias.

However, alias substitution should not be performed on a reserved word in any case. It is your responsibility to check the token type and not to call this function on a reserved word. That is why this function is named manual. To consume a reserved word without performing alias substitution, you should call take_token_raw or take_token_auto.

pub async fn take_token_auto(&mut self, keywords: &[Keyword]) -> Result<Token>

Consumes the current token after performing applicable alias substitution.

This function performs alias substitution unless the result is one of the reserved words specified in the argument.

Alias substitution is performed repeatedly until a non-alias token is found. That is why this function is named auto. This function should be used only in contexts where no backtrack is needed after alias substitution. If you need to backtrack or want to know whether alias substitution was performed or not, you should use Self::take_token_manual, which performs alias substitution at most once and returns Rec.

pub async fn has_blank(&mut self) -> Result<bool>

Tests if there is a blank before the next token.

This function can be called to tell whether the previous and next tokens are separated by a blank or they are adjacent.

This function must be called after the previous token has been taken (by one of take_token_raw, take_token_manual and take_token_auto) and before the next token is peeked. Otherwise, this function would panic.

Panics

If the previous token has not been taken or the next token has been peeked.

pub fn memorize_unread_here_doc(&mut self, here_doc: Rc<HereDoc>)

Remembers the given partial here-document for later parsing of its content.

The remembered here-document’s content will be parsed when here_doc_contents is called later.

pub async fn here_doc_contents(&mut self) -> Result<()>

Reads here-document contents that matches the remembered list of here-document operators.

This function reads here-document contents corresponding to here-document operators that have been saved with memorize_unread_here_doc. The results are inserted to the content field of the HereDoc instances.

This function must be called just after a newline token has been taken (either manual or auto). If there is a pending token that has been peeked but not yet taken, this function will panic!

pub fn ensure_no_unread_here_doc(&self) -> Result<()>

Ensures that there is no pending partial here-document.

If there is any, this function returns a MissingHereDocContent error.

impl Parser<'_, '_>

pub async fn and_or_list(&mut self) -> Result<Rec<Option<AndOrList>>>

Parses an and-or list.

If there is no valid and-or list at the current position, this function returns Ok(Rec::Parsed(None)).

impl Parser<'_, '_>

pub async fn case_item(&mut self) -> Result<Option<(CaseItem, bool)>>

Parses a case item.

If the next token is esac, returns None without consuming it. Otherwise, returns a case item and whether there may be a next item.

pub async fn case_command(&mut self) -> Result<CompoundCommand>

Parses a case conditional construct.

The next token must be the case reserved word.

Panics

If the first token is not case.

impl Parser<'_, '_>

pub async fn command(&mut self) -> Result<Rec<Option<Command>>>

Parses a command.

If there is no valid command at the current position, this function returns Ok(Rec::Parsed(None)).

impl Parser<'_, '_>

pub async fn do_clause(&mut self) -> Result<Option<List>>

Parses a do clause, i.e., a compound list surrounded in do ... done.

Returns Ok(None) if the first token is not do.

pub async fn compound_command(&mut self) -> Result<Option<CompoundCommand>>

Parses a compound command.

pub async fn full_compound_command( &mut self, ) -> Result<Option<FullCompoundCommand>>

Parses a compound command with optional redirections.

impl Parser<'_, '_>

pub async fn for_loop(&mut self) -> Result<CompoundCommand>

Parses a for loop.

The next token must be the for reserved word.

Panics

If the first token is not for.

impl Parser<'_, '_>

pub async fn short_function_definition( &mut self, intro: SimpleCommand, ) -> Result<Command>

Parses a function definition command that does not start with the function reserved word.

This function must be called just after a simple command has been parsed. The simple command must be passed as an argument. If the simple command has only one word and the next token is (, it is parsed as a function definition command. Otherwise, the simple command is returned intact.

impl Parser<'_, '_>

pub async fn grouping(&mut self) -> Result<CompoundCommand>

Parses a normal grouping.

The next token must be a {.

Panics

If the first token is not a {.

pub async fn subshell(&mut self) -> Result<CompoundCommand>

Parses a subshell.

The next token must be a (.

Panics

If the first token is not a (.

impl Parser<'_, '_>

pub async fn if_command(&mut self) -> Result<CompoundCommand>

Parses an if conditional construct.

The next token must be the if reserved word.

Panics

If the first token is not if.

impl Parser<'_, '_>

pub async fn list(&mut self) -> Result<Rec<List>>

Parses a list.

This function parses a sequence of and-or lists that are separated by ; or &. A newline token that delimits the list is not parsed.

If there is no valid command at the current position, this function returns a list with no items.

pub async fn newline_and_here_doc_contents(&mut self) -> Result<bool>

Parses an optional newline token and here-document contents.

If the current token is a newline, it is consumed and any pending here-document contents are read starting from the next line. Otherwise, this function returns Ok(false) without any side effect.

pub async fn command_line(&mut self) -> Result<Option<List>>

Parses a complete command optionally delimited by a newline.

A complete command is a minimal sequence of and-or lists that can be executed in the shell environment. This function reads as many lines as needed to compose the complete command.

If the current line is empty (or containing only whitespaces and comments), the result is an empty list. If the first token of the current line is the end of input, the result is Ok(None).

pub async fn maybe_compound_list(&mut self) -> Result<List>

Parses an optional compound list.

A compound list is a sequence of one or more and-or lists that are separated by newlines and optionally preceded and/or followed by newlines.

This function stops parsing on encountering an unexpected token that cannot be parsed as the beginning of an and-or list. If the token is a possible clause delimiter, the result is a list of commands that have been parsed up to the token. Otherwise, an InvalidCommandToken error is returned.

pub fn maybe_compound_list_boxed( &mut self, ) -> Pin<Box<dyn Future<Output = Result<List>> + '_>>

Like maybe_compound_list, but returns the future in a pinning box.

impl Parser<'_, '_>

pub async fn pipeline(&mut self) -> Result<Rec<Option<Pipeline>>>

Parses a pipeline.

If there is no valid pipeline at the current position, this function returns Ok(Rec::Parsed(None)).

impl Parser<'_, '_>

pub async fn redirection(&mut self) -> Result<Option<Redir>>

Parses a redirection.

If the current token is not a redirection operator, Ok(None) is returned. If a word token is missing after the operator, Err(Error{...}) is returned with a cause of MissingRedirOperand or MissingHereDocDelimiter.

pub async fn redirections(&mut self) -> Result<Vec<Redir>>

Parses a (possibly empty) sequence of redirections.

impl Parser<'_, '_>

pub async fn array_values(&mut self) -> Result<Option<Vec<Word>>>

Parses the value of an array assignment.

This function first consumes a ( token, then any number of words separated by blanks and/or newlines, and finally a ). If the first token is not (, the result is Ok(None). If the last ) is missing, the result is Err(ErrorCause::Syntax(SyntaxError::UnclosedArrayValue(_))).

pub async fn simple_command(&mut self) -> Result<Rec<Option<SimpleCommand>>>

Parses a simple command.

If there is no valid command at the current position, this function returns Ok(Rec::Parsed(None)).

impl Parser<'_, '_>

pub async fn while_loop(&mut self) -> Result<CompoundCommand>

Parses a while loop.

The next token must be the while reserved word.

Panics

If the first token is not while.

pub async fn until_loop(&mut self) -> Result<CompoundCommand>

Parses an until loop.

The next token must be the until reserved word.

Panics

If the first token is not until.

Trait Implementations

impl<'a, 'b> Debug for Parser<'a, 'b>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.