Module streaming 

Streaming decode → scan adapter. Fuses a decoder Program with a scanner Program so decoded bytes hand off through workgroup-shared memory instead of a DRAM round-trip. Streaming decode → scan adapter (G5).

What this does

decode::base64 / hex / inflate / lz4 each used to produce a storage-buffer output that matching::dfa / nfa then re-read from DRAM. Two kernels, two DMA round-trips, one pipeline barrier - cheap on a 1 MiB corpus, ruinous on a 100 GiB corpus scan where the decoded-bytes footprint dominates the DRAM budget.

The fused path hands bytes from decoder → scanner through workgroup-shared memory on the same dispatch. No DRAM round-trip, no pipeline barrier - the scanner’s loads hit L1 on the SM that decoded the bytes.

Contract

The caller supplies a decoder Program that writes a named handoff buffer and a scanner Program that reads the same named buffer. fuse_decode_scan merges them via the existing vyre_foundation::execution_plan::fusion::fuse_programs kernel fuser, then rewrites the handoff buffer’s declaration so it lives in workgroup memory instead of storage.

Why this module lives in vyre-libs

The fusion transformation itself is a foundation-layer pass (optimizer::passes::decode_scan_fuse). This module is the library-level API that consumers reach for directly. Both paths land on the same fused Program - the foundation pass is the canonical transformation, this is a thin convenience layer.

Enums

DecodeScanFuseError

Error states surfaced by fuse_decode_scan.

Functions

dram_bytes_saved

How many bytes of DRAM traffic one dispatch saves by fusing a decoder+scanner pair with an N-byte handoff. Used by the G12 benchmark harness to verify the fusion is paying off.

fuse_decode_scan

Fuse a decoder Program with a scanner Program into a single dispatch. The decoder writes handoff_buf; the scanner reads it. In the fused Program the handoff buffer is promoted to workgroup memory so its bytes never touch DRAM.