Module lexing

Modules

errors

Structs

Token

A token is a single unit of a command, such as a word, number or symbol. This is used to convert single characters in a more machine-readable format. For example, the string “ls -l” would be converted to a list of tokens like so: [“ls”, “-l”].

Enums

TokenKind

As we are only implementing a very simple shell, we only need a few token kinds. These are the token kinds we are going to use:

Functions

tokenise