Function tokenize_str_with_options 

pub fn tokenize_str_with_options(
    input: &str,
    options: &TokenizerOptions,
) -> Result<Vec<Token>, TokenizerError>

Break the given input shell script string into tokens, returning the tokens.

Arguments

• input - The shell script to tokenize.
• options - Options controlling how the tokenizer operates.