Module constants 

Compile-time constants: suffix lists, punctuation sets.

Constants

AFFIX_SPACING_PATTERNS

(pattern, replacement) pairs for attaching prefixes/suffixes with ZWNJ.

DIACRITICS_PATTERN

Regex pattern that matches Arabic diacritical marks to be removed.

EMAIL_PATTERN

Pattern for e-mail addresses.

EMOJI_PATTERN

Emoji range pattern (covers major Unicode emoji blocks).

EXTRA_SPACE_PATTERNS

(pattern, replacement) pairs for removing redundant whitespace and ZWNJ.

HASHTAG_PATTERN

Pattern for hashtags.

ID_PATTERN

Pattern for Social-network @mentions.

LINK_PATTERN

Pattern for URLs/links.

MORE_THAN_TWO_REPEAT_PATTERN

Matches any Persian letter repeated 3+ times in a row.

NUMBERS_DST

Persian equivalents used to replace NUMBERS_SRC characters.

NUMBERS_SRC

Source characters for digit conversion (ASCII/Arabic → Persian).

NUMBER_FLOAT_PATTERN

Pattern for floating-point numbers.

NUMBER_INT_PATTERN

Pattern for integer numbers (both ASCII and Persian digits).

PERSIAN_LETTERS

All standard Persian/Arabic letters, used to build character class patterns.

PERSIAN_STYLE_PATTERNS

(pattern, replacement) pairs for Persian typographic style.

PUNCTUATION_SPACING_PATTERNS

(pattern, replacement) pairs for normalizing spacing around punctuation.

REPEATED_CHARS_PATTERN

Matches entire words that contain a Persian letter repeated 3+ times.

SEPERATE_MI_PATTERN

Pattern used in seperate_mi — matches می/نمی attached to a verb stem.

SPECIAL_CHARS_PATTERN

Regex pattern matching rare/decorative Unicode code points to be stripped.

SUFFIXES

Inflectional and derivational suffixes stripped by the rule-based stemmer.

TRANSLATION_DST

Destination characters for the Unicode normalization table.

TRANSLATION_SRC

Source characters for Unicode normalization (maps variant forms to standard Persian).

UNICODE_REPLACEMENTS

Pairs of (ligature, replacement text) for common Arabic/Persian Unicode ligatures.

WORD_SPLIT_PATTERN

Main tokenization split pattern — splits on punctuation, brackets, digits.