Module mode 

Tokenization modes and penalty configurations.

This module defines the different tokenization modes available and their penalty configurations for controlling segmentation behavior.

Structs

JsPenalty

Penalty configuration for decompose mode.

Enums

JsMode

Tokenization mode.