An atom is a portion of text within a document that is significant.
An atom only covers one line.
Atoms cover keywords as well as start and end indicators for bounded tokens
E.g., in a string, the atoms would be the starting “ and the ending “
Definition for a bounded token, these are tokens that can cover multiple lines.
Things like multiline comments and strings are examples of this.
They work well for buffering files where you are unaware of where the end indicator may be as
it occurs further down in the file.