Expand description
The transformer pipeline and transformation traits.
In Tokel, a transformer is a component that maps an input token stream
to an output token stream. Transformers are applied sequentially to
the fully resolved output of an expansion block ([< ... >]).
This module defines the core Transformer trait that all built-in (and potentially
custom) token manipulators must implement, along with standard utility transformers
like Identity.
Structs§
- Identity
- An identity
Transformer. - Registry
- A centralized registry of
Transformerof distinct types.
Traits§
- Pass
- A trait that represents a singular pass over an input token stream.
- Transformer
- A transformer in a pipeline.