lexical_scanner 0.1.13

A simple lexer which creates over 75 various tokens based on the rust programming language. This complete Lexer/Lexical Scanner produces tokens for a string or a file path entry. The output is a Vector<Token> for the user to handle according to their needs. All tokens are included (including whitespace) as that is left to the user to decide how to use the tokens. This library does not use regex!