Struct yash_syntax::parser::Parser

pub struct Parser<'a, 'b> { /* private fields */ }

The shell syntax parser.

This struct contains a set of data used in syntax parsing.

Because of the nature of here-document contents that cannot be parsed by a standard recursive descent parser, most intrinsic functions of Parser return an AST with MissingHereDoc filled in place of a proper here-document data. When the parser parses a here-document content, the content is accumulated in the parser so that it can be merged into the main AST in the final step of parsing.

Unless you are interested in parsing a specific syntactic construct that is only part of source code, you will want to use a function that returns a complete result filled with proper here-document contents if any. Then the command_line function is for you. See also the module documentation.

Implementations

impl<'a, 'b> Parser<'a, 'b>

pub fn new(lexer: &'a mut Lexer<'b>, aliases: &'a AliasSet) -> Parser<'a, 'b>

Creates a new parser based on the given lexer and alias set.

pub async fn peek_token(&mut self) -> Result<&Token>

Returns a reference to the current token.

If the current token is not yet read from the underlying lexer, it is read.

pub async fn take_token_raw(&mut self) -> Result<Token>

Consumes the current token without performing alias substitution.

If the current token is not yet read from the underlying lexer, it is read.

This function does not perform alias substitution and therefore should be used only in context where no alias substitution is expected. Otherwise, you should use take_token_manual or take_token_auto instead.

pub async fn take_token_manual(
    &mut self,
    is_command_name: bool
) -> Result<Rec<Token>>

Consumes the current token after performing applicable alias substitution.

If the current token is not yet read from the underlying lexer, it is read.

This function checks if the token is the name of an alias. If it is, alias substitution is performed on the token and the result is Ok(AliasSubstituted). Otherwise, the token is consumed and returned.

Alias substitution is performed only if at least one of the following is true:

• The token is the first command word in a simple command, that is, it is the word for the command name. (This condition should be specified by the is_command_name parameter.)
• The token comes just after the replacement string of another alias substitution that ends with a blank character.
• The token names a global alias.

However, alias substitution should not be performed on a reserved word in any case. It is your responsibility to check the token type and not to call this function on a reserved word. That is why this function is named manual. To consume a reserved word without performing alias substitution, you should call take_token_raw or take_token_auto.

pub async fn take_token_auto(&mut self, keywords: &[Keyword]) -> Result<Token>

Consumes the current token after performing applicable alias substitution.

This function performs alias substitution unless the result is one of the reserved words specified in the argument.

Alias substitution is performed repeatedly until a non-alias token is found. That is why this function is named auto. This function should be used only in contexts where no backtrack is needed after alias substitution. If you need to backtrack or want to know whether alias substitution was performed or not, you should use Self::take_token_manual, which performs alias substitution at most once and returns Rec.

pub async fn has_blank(&mut self) -> Result<bool>

Tests if there is a blank before the next token.

This function can be called to tell whether the previous and next tokens are separated by a blank or they are adjacent.

This function must be called after the previous token has been taken (by one of take_token_raw, take_token_manual and take_token_auto) and before the next token is peeked. Otherwise, this function would panic.

Panics

If the previous token has not been taken or the next token has been peeked.

pub fn memorize_unread_here_doc(&mut self, here_doc: PartialHereDoc)

Remembers the given partial here-document for later parsing of its content.

pub async fn here_doc_contents(&mut self) -> Result<()>

Reads here-document contents that matches the remembered list of partial here-documents.

The results are accumulated in the internal list of (non-partial) here-documents.

This function must be called just after a newline token has been taken (either manual or auto). If there is a pending token that has been peeked but not yet taken, this function will panic!

pub fn ensure_no_unread_here_doc(&self) -> Result<()>

Ensures that there is no pending partial here-document.

If there is any, this function returns a MissingHereDocContent error.

pub fn take_read_here_docs(&mut self) -> Vec<HereDoc>

Returns a list of here-documents with contents that have been read.

impl Parser<'_, '_>

pub async fn and_or_list(
    &mut self
) -> Result<Rec<Option<AndOrList<MissingHereDoc>>>>

Parses an and-or list.

If there is no valid and-or list at the current position, this function returns Ok(Rec::Parsed(None)).

impl Parser<'_, '_>

pub async fn case_item(&mut self) -> Result<Option<CaseItem<MissingHereDoc>>>

Parses a case item.

Does not parse the optional trailing double semicolon.

Returns None if the next token is esac.

pub async fn case_command(&mut self) -> Result<CompoundCommand<MissingHereDoc>>

Parses a case conditional construct.

The next token must be the case reserved word.

Panics

If the first token is not case.

impl Parser<'_, '_>

pub async fn command(&mut self) -> Result<Rec<Option<Command<MissingHereDoc>>>>

Parses a command.

If there is no valid command at the current position, this function returns Ok(Rec::Parsed(None)).

impl Parser<'_, '_>

pub async fn do_clause(&mut self) -> Result<Option<List<MissingHereDoc>>>

Parses a do clause, i.e., a compound list surrounded in do ... done.

Returns Ok(None) if the first token is not do.

pub async fn compound_command(
    &mut self
) -> Result<Option<CompoundCommand<MissingHereDoc>>>

Parses a compound command.

pub async fn full_compound_command(
    &mut self
) -> Result<Option<FullCompoundCommand<MissingHereDoc>>>

Parses a compound command with optional redirections.

impl Parser<'_, '_>

pub async fn for_loop(&mut self) -> Result<CompoundCommand<MissingHereDoc>>

Parses a for loop.

The next token must be the for reserved word.

Panics

If the first token is not for.

impl Parser<'_, '_>

pub async fn short_function_definition(
    &mut self,
    intro: SimpleCommand<MissingHereDoc>
) -> Result<Command<MissingHereDoc>>

Parses a function definition command that does not start with the function reserved word.

This function must be called just after a simple command has been parsed. The simple command must be passed as an argument. If the simple command has only one word and the next token is (, it is parsed as a function definition command. Otherwise, the simple command is returned intact.

impl Parser<'_, '_>

pub async fn grouping(&mut self) -> Result<CompoundCommand<MissingHereDoc>>

Parses a normal grouping.

The next token must be a {.

Panics

If the first token is not a {.

pub async fn subshell(&mut self) -> Result<CompoundCommand<MissingHereDoc>>

Parses a subshell.

The next token must be a (.

Panics

If the first token is not a (.

impl Parser<'_, '_>

pub async fn if_command(&mut self) -> Result<CompoundCommand<MissingHereDoc>>

Parses an if conditional construct.

The next token must be the if reserved word.

Panics

If the first token is not if.

impl Parser<'_, '_>

pub async fn list(&mut self) -> Result<Rec<List<MissingHereDoc>>>

Parses a list.

This function parses a sequence of and-or lists that are separated by ; or &. A newline token that delimits the list is not parsed.

If there is no valid command at the current position, this function returns a list with no items.

pub async fn newline_and_here_doc_contents(&mut self) -> Result<bool>

Parses an optional newline token and here-document contents.

If the current token is a newline, it is consumed and any pending here-document contents are read starting from the next line. Otherwise, this function returns Ok(false) without any side effect.

pub async fn command_line(&mut self) -> Result<Option<List>>

Parses a complete command optionally delimited by a newline.

A complete command is a minimal sequence of and-or lists that can be executed in the shell environment. This function reads as many lines as needed to compose the complete command.

If the current line is empty (or containing only whitespaces and comments), the result is an empty list. If the first token of the current line is the end of input, the result is Ok(None).

pub async fn maybe_compound_list(&mut self) -> Result<List<MissingHereDoc>>

Parses an optional compound list.

A compound list is a sequence of one or more and-or lists that are separated by newlines and optionally preceded and/or followed by newlines.

This function stops parsing on encountering an unexpected token that cannot be parsed as the beginning of an and-or list. The caller should check that the next token is an expected one.

pub fn maybe_compound_list_boxed(
    &mut self
) -> Pin<Box<dyn Future<Output = Result<List<MissingHereDoc>>>>>

Like maybe_compound_list, but returns the future in a pinned box.

impl Parser<'_, '_>

pub async fn pipeline(
    &mut self
) -> Result<Rec<Option<Pipeline<MissingHereDoc>>>>

Parses a pipeline.

If there is no valid pipeline at the current position, this function returns Ok(Rec::Parsed(None)).

impl Parser<'_, '_>

pub async fn redirection(&mut self) -> Result<Option<Redir<MissingHereDoc>>>

Parses a redirection.

If the current token is not a redirection operator, Ok(None) is returned. If a word token is missing after the operator, Err(Error{...}) is returned with a cause of MissingRedirOperand or MissingHereDocDelimiter.

pub async fn redirections(&mut self) -> Result<Vec<Redir<MissingHereDoc>>>

Parses a (possibly empty) sequence of redirections.

impl Parser<'_, '_>

pub async fn array_values(&mut self) -> Result<Option<Vec<Word>>>

Parses the value of an array assignment.

This function first consumes a ( token, then any number of words separated by blanks and/or newlines, and finally a ). If the first token is not (, the result is Ok(None). If the last ) is missing, the result is Err(ErrorCause::Syntax(SyntaxError::UnclosedArrayValue(_))).

pub async fn simple_command(
    &mut self
) -> Result<Rec<Option<SimpleCommand<MissingHereDoc>>>>

Parses a simple command.

If there is no valid command at the current position, this function returns Ok(Rec::Parsed(None)).

impl Parser<'_, '_>

pub async fn while_loop(&mut self) -> Result<CompoundCommand<MissingHereDoc>>

Parses a while loop.

The next token must be the while reserved word.

Panics

If the first token is not while.

pub async fn until_loop(&mut self) -> Result<CompoundCommand<MissingHereDoc>>

Parses an until loop.

The next token must be the until reserved word.

Panics

If the first token is not until.

Trait Implementations

impl<'a, 'b> Debug for Parser<'a, 'b>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.