Skip to main content

tokenize

latex_rust::parser

Function tokenize 

Source 
pub fn tokenize(input: &str) -> Result<Vec<Token>, ParseError>
Expand description

Tokenize a LaTeX math string. Whitespace and % line comments are skipped.

Supported delimiters are tokenized, not interpreted: $...$, $$...$$, \[...\], \(...\).

§Arguments

  • input — math source.

§Returns

The token list in source order.

§Errors

ParseError::TrailingBackslash if input ends with a stray \.

§Examples

use latex_rust::{tokenize, Token};

let t = tokenize(r"\frac{1}{2}").unwrap();
assert!(matches!(&t[0], Token::Command(s) if s == "frac"));