Module simd 

SIMD-friendly multi-byte search primitives.

Pure-safe Rust (no unsafe, no platform intrinsics, no hardware-specific deps). The vectorisation comes from memchr’s SSE2 / NEON dispatch for arity 1/2/3 and SWAR (SIMD-Within-A-Register) for arity 4+. The parser hot path uses these primitives unconditionally; the simd Cargo feature is retained as a no-op for forward compatibility. SIMD-friendly structural-scanning primitives.

YAML’s parser hot paths are dominated by multi-byte search — “find the next byte that’s any of [':', ',', '[', ']', '{', '}', '\\n', '#']”, “find the next non-blank byte”, “find the end of a plain scalar”. These searches dwarf the structural work for typical inputs because every byte of every scalar flows through them.

This module ships a feature-gated primitive surface for that scanning, with three implementation strata:

• arity 1, 2, 3 — delegate to the memchr crate, which already compiles to SSE2 on x86_64 and NEON on aarch64. We gain nothing by hand-rolling.
• arity 4+ — pack 8 bytes per u64 lane and use SWAR (SIMD Within A Register) to test all bytes in parallel against the needle bitmap. Beats byte-by-byte by ~3-5× on needle sets up to 16 bytes wide.
• fallback — a portable scalar loop that the optimiser auto-vectorises on most LLVM targets.

Pure-safe Rust — every primitive compiles under #![forbid(unsafe_code)]. The implementation does not call core::arch::* intrinsics directly; it lets memchr’s already- validated unsafe abstractions do the platform dispatch and uses pure SWAR bit-twiddling for the wider-arity cases.

Examples

use noyalib::simd::find_any_of;

let haystack = b"some plain text: with a colon";
// Find the first occurrence of any of these terminator bytes.
let pos = find_any_of(haystack, &[b':', b',', b'\n']);
assert_eq!(pos, Some(15));

use noyalib::simd::find_any_of;

// Returns None when no needle is present.
assert_eq!(find_any_of(b"abcdef", &[b'X', b'Y', b'Z']), None);

Structs

ByteBitmap

256-bit bitmap of byte values, used by find_byte_in_bitmap. Construct via bitmap_for.

SimdScanner

Build-once, scan-many structural scanner.

StructuralIter

Iterator over every structural-byte position in a haystack.

Constants

BLOCK_PLAIN_NEEDLES

Canonical YAML 1.2 plain-scalar boundary candidate set in block context. The scanner consumes a plain scalar by “jumping” to the first byte in this set, then validating whether it is a true terminator (e.g. : is only a key indicator when followed by whitespace, # is only a comment when preceded by whitespace).

FLOW_PLAIN_NEEDLES

Canonical YAML 1.2 plain-scalar boundary candidate set inside flow collections ([ ], { }). Adds the flow-collection terminators (,, [, ], {, }) to the block set.

LINE_BREAK_NEEDLES

Canonical newline-discovery needles. Used by block-scalar (| / >) line counting, comment-text scanning, and any other “find next line break” hot path.

Functions

bitmap_for

Build a ByteBitmap from a list of bytes. Duplicate entries are coalesced (idempotent set semantics).

clean_prefix_len

Length of the leading run of bytes in haystack that are NOT in needles. Equivalent to find_any_of(haystack, needles).unwrap_or(haystack.len()) but expresses the “skip-clean-prefix” intent more directly at call sites. Useful in scanner inner loops where the next event is “consume the run, then handle the boundary byte”.

find_any_of

Find the byte offset of the first occurrence of any byte in needles within haystack.

find_byte_in_bitmap

Find the byte offset of the first occurrence of any byte whose value is set in bitmap. The bitmap encodes a 256-bit set — bit b is set ⇔ byte value b is a needle.

match_prefix_len

Length of the leading run of bytes in haystack that ARE in needles. The polarity-flipped sibling of clean_prefix_len:

parse_decimal_i64

SWAR decimal parser for signed i64. Accepts an optional leading + or -, then forwards to parse_decimal_u64. Returns None for empty input, non-digit bytes, or overflow in either direction.

parse_decimal_u64

SWAR decimal parser: parse an unsigned 64-bit integer from a byte slice of ASCII digits. Returns None if the slice is empty, contains a non-digit byte, or overflows u64.