Module tokenize 

EDIFACT tokenization helper — collects segments from input into a Vec.

The edifact-parser crate uses an event-driven (SAX-style) API. This module provides a convenience function that collects all parsed segments into an owned Vec<OwnedSegment> for two-pass processing.

OwnedSegment itself lives in mig-types::segment — re-exported here for backward compatibility.

Structs

InterchangeChunks

A complete EDIFACT interchange split into per-message chunks.

MessageChunk

A single EDIFACT message (UNH…UNT) with its interchange envelope.

OwnedSegment

An owned version of a parsed EDIFACT segment — stores String data.

Functions

parse_to_segments

Parse an EDIFACT message into a list of owned segments.

split_messages

Split a flat segment list into per-message chunks at UNH/UNT boundaries.