Module tokenizer

Block and field tokenizer for SWIFT MT messages

This module handles the extraction of SWIFT message blocks and parsing of individual fields from Block 4 (Text Block). It provides efficient tokenization with proper error handling and supports streaming for large messages.

Structs

ApplicationHeader

Application Header (Block 2) structure

BalanceCheckpoint

Balance checkpoint structure for Tag 423

BasicHeader

Basic Header (Block 1) structure

MessageInputReference

Message Input Reference structure for Tag 106

MessageOutputReference

Message Output Reference structure (similar to MIR but for output)

MessageReference

Message Reference structure for MRF tag

ParsedField

Parsed field structure

PaymentControlsInfo

Payment controls information structure for Tag 434

PaymentReleaseInfo

Payment release information structure for Tag 165

PossibleDuplicateEmission

Possible Duplicate Emission structure for PDE tag

PossibleDuplicateMessage

Possible Duplicate Message structure for PDM tag

SanctionsScreeningInfo

Sanctions screening information structure for Tag 433

SwiftMessageBlocks

SWIFT message blocks structure

SystemOriginatedMessage

System Originated Message structure for SYS tag

Trailer

Trailer (Block 5) structure based on SWIFT MT standards

UserHeader

User Header (Block 3) structure based on SWIFT MT standards

Functions

extract_blocks

Extract blocks from a SWIFT message using proper brace matching

extract_message_type

Extract message type from blocks

parse_application_header

Parse application header from Block 2

parse_basic_header

Parse basic header from Block 1

parse_block4_fields

Parse Block 4 fields into individual field structures

parse_trailer_block

Parse trailer block from Block 5 using structured parsing

parse_user_header

Parse user header from Block 3 using structured parsing