Function blarse::str_to_tokens

pub fn str_to_tokens<'a>(body: &'a str) -> Vec<Token<'a>, Global>

Chops up a string slice into a vector of owned tokens. Also appends an empty token to the tail of the vector to enable certain lexing functions like scanning for words.