[][src]Function ruby_parser::tokenize

pub fn tokenize(i: Input<'_>) -> IResult<Input<'_>, Vec<Token>>

Tokenizes a ruby program