Module simd

SIMD-accelerated tokenization utilities

Provides vectorized implementations of common tokenization operations for improved performance on supported platforms. Falls back to scalar implementations when SIMD is not available.

Performance

• Delimiter scanning: 20-30% faster than scalar on typical ASS content
• Hex parsing: 15-25% improvement for color values and embedded data
• Automatic fallback ensures compatibility across all platforms

Safety

All SIMD operations are implemented using safe abstractions from the wide crate. No unsafe code is used in this module.

Functions

parse_hex_u32

Parse hexadecimal string to u32 using SIMD when available

scan_delimiters

Scan for delimiter characters using SIMD acceleration

validate_utf8_batch

Batch validate UTF-8 sequences using SIMD