pub fn tokenize_algol_spanned(source: &str) -> Result<Vec<SpannedToken>>Expand description
Tokenizes Algol source into spanned tokens under a default decode budget.
pub fn tokenize_algol_spanned(source: &str) -> Result<Vec<SpannedToken>>Tokenizes Algol source into spanned tokens under a default decode budget.