Module tokenizer 

Functions

is_arrow

Returns true if the token is an arrow operator: ->, <->, or –

is_key_value

Returns true if the token is a key:value pair (not a selector, not an arrow).

is_selector

Returns true if the token is a selector (starts with @).

parse_key_value

Parses a key:value token into its key and value parts. If the value is wrapped in double quotes, they are stripped.

parse_key_value_with_meta

Parses a key:value token, also returning whether the value was quoted.

tokenize

Tokenize splits an FCP operation string into tokens, handling quoted strings, escape sequences, and embedded newlines.