Function tokenize_str_with_options 

pub fn tokenize_str_with_options(
    input: &str,
    options: &TokenizerOptions,
) -> Result<Vec<Token>, TokenizerError>

Break the given input shell script string into tokens, returning the tokens.

Arguments

• input - The shell script to tokenize.
• options - Options controlling how the tokenizer operates.

Errors

• crate::TokenizerError::UnterminatedEscapeSequence, crate::TokenizerError::UnterminatedSingleQuote, crate::TokenizerError::UnterminatedAnsiCQuote, crate::TokenizerError::UnterminatedDoubleQuote, crate::TokenizerError::UnterminatedBackquote or crate::TokenizerError::UnterminatedExtendedGlob if the input ends inside the matching quoting construct.
• crate::TokenizerError::UnterminatedVariable, crate::TokenizerError::UnterminatedCommandSubstitution or crate::TokenizerError::UnterminatedExpansion if the input ends inside a ${...}, $(...), $((...)) or $[...] expansion.
• crate::TokenizerError::UnterminatedHereDocuments if the input ends before the body of a declared here-document.
• crate::TokenizerError::MissingHereTag or crate::TokenizerError::MissingHereTagAtEndOfInput if a << or <<- operator has no tag.
• crate::TokenizerError::ExpansionNestingTooDeep if expansions nest more than 32 levels deep.