Struct StructuralIndexer 

pub struct StructuralIndexer { /* private fields */ }

SIMD-powered structural character indexer.

Scans input in 64-byte blocks, producing bitmasks for each delimiter type. Quote-aware masking ensures delimiters inside "..." or '...' are not flagged as structural.

Example

use fhp_tokenizer::structural::StructuralIndexer;

let indexer = StructuralIndexer::new();
let index = indexer.index(b"<div class=\"foo\">bar</div>");

let delimiters: Vec<_> = index.iter_delimiters().collect();
assert!(delimiters.len() > 0);

Implementations

impl StructuralIndexer

pub fn new() -> Self

Create a new indexer using the auto-detected SIMD backend.

pub fn index(&self, input: &[u8]) -> StructuralIndex

Scan input and produce a StructuralIndex.

The input is processed in 64-byte blocks. For each block, a BlockBitmaps is produced with bitmasks for every HTML delimiter. Quote-aware masking is then applied to clear non-structural positions.

Trait Implementations

impl Default for StructuralIndexer

fn default() -> Self

Returns the “default value” for a type.