memspan 0.1.0

SIMD-accelerated byte-class scanning for lexers and parsers.

English | 简体中文

Quick start

let src = b"   hello, world";

// Skip leading whitespace — dispatches to AVX2 / NEON / SIMD128 at runtime.
let n = memspan::skip_whitespace(src);
assert_eq!(n, 3);

// Find the first comma.
let comma = memspan::skip_until(src, b',');
assert_eq!(comma, Some(8));

Overview

memspan provides zero-allocation, no_std-compatible functions to skip, count, and locate bytes in ASCII character classes, dispatching to the best available SIMD backend at runtime:

Installation

[dependencies]
memspan = "0.1"

For no_std without an allocator:

[dependencies]
memspan = { version = "0.1", default-features = false }

Built-in classes

All class functions return the byte length of the longest matching prefix.

Generic operations

skip_while, skip_until, count_matches, and find_last accept any [Needles] value — a single u8, a fixed-size array [u8; N], or a &[u8] slice:

// Skip while any of several bytes match.
let n = memspan::skip_while(b"  ,\t ok", [b' ', b',', b'\t']);
assert_eq!(n, 5);

// Find the first occurrence of any needle — like memchr but multi-byte.
let pos = memspan::skip_until(b"hello\nworld", b'\n');
assert_eq!(pos, Some(5));

// Count every newline in a buffer.
let lines = memspan::count_matches(b"a\nb\nc\n", b'\n');
assert_eq!(lines, 3);

// Find the rightmost match.
let last = memspan::find_last(b"\"hello\"", b'"');
assert_eq!(last, Some(6));

Custom classes with skip_class!

Define your own byte class and get the same SIMD dispatch as the built-ins:

memspan::skip_class! {
    /// Skip whitespace and commas.
    pub fn skip_ws_and_comma(bytes = [b' ', b'\t', b'\r', b'\n', b',']);
}

memspan::skip_class! {
    /// Skip lowercase ASCII letters.
    pub fn skip_lowercase(ranges = [b'a'..=b'z']);
}

memspan::skip_class! {
    /// Skip alphanumeric plus common punctuation.
    pub fn skip_punct_ident(
        bytes  = [b'_', b'-', b'!', b'?'],
        ranges = [b'a'..=b'z', b'A'..=b'Z', b'0'..=b'9'],
    );
}

assert_eq!(skip_ws_and_comma(b"  , ok"), 4);
assert_eq!(skip_lowercase(b"abcXYZ"), 3);
assert_eq!(skip_punct_ident(b"hello-world! 42"), 12);

Benchmarks

Throughput in GiB/s across input sizes, measured on GitHub Actions runners (2026-04-22, --quick Criterion runs). Environments: aarch64 — macOS-latest (ARM64, NEON); x86_64 — ubuntu-latest (X64, runtime AVX2 detection).

aarch64 — NEON

aarch64 — scalar fallback

x86_64 — AVX2

x86_64 — scalar fallback

Generic dispatch (skip_until / skip_while)

skip_class! macro vs skip_while

Features

License

memspan is dual-licensed under the MIT license and the Apache License (Version 2.0).

See LICENSE-APACHE, LICENSE-MIT for details.

Copyright (c) 2026 Al Liu.