Skip to main content

Crate rusty_json_turbo_accel

Crate rusty_json_turbo_accel 

Source
Expand description

The SIMD island: SSE2 and AVX2 twins of the JSON byte scanners.

§Why this is a separate crate

Vector intrinsics are unsafe, and the workspace denies unsafe_code everywhere else. Rather than open a hole in the parser, the vector code lives here, alone, so rusty_json_turbo can keep tightening toward forbid(unsafe_code) while still reaching a wide scan. Every unsafe in this crate is a load or a compare, each one bounds-checked by the caller immediately above it and carrying its invariant in a SAFETY comment.

There are no dependencies and no build script. That is a contract, not an accident: rusty_json_turbo advertises an empty dependency tree.

§The rule every kernel here follows

The scalar version is written first, it is the oracle, and it stays in the tree forever. Each vector kernel is tested against it over every one of the 256 byte values at every offset, and RJT_ISA=scalar runs it in production so the two can be compared inside one binary. A vector kernel whose scalar twin has been deleted cannot be checked and cannot be debugged.

§What the corpus said before any of this was written

Sizing a kernel is a measurement, not a preference for the widest register available. On JSONCORP, whitespace runs are 16 to 29 bytes on citm_catalog and nothing in the corpus reaches 32 bytes, so no 32-byte step is ever fully used. That does not make AVX2 worthless – a wide load that overshoots the end of a run still resolves it in one iteration – but it does mean the honest figure is steps taken, not bytes covered: 8-byte steps 140,381, 16-byte 95,424, 32-byte 50,467 on citm_catalog, and 16,252 against 15,476 on twitter, which is nothing.

So the whitespace kernel is a citm_catalog win and twitter is a CONTROL. That distinction was paid for once already: brick B1s cost twitter 10% by taking an 8-byte step over its 1-byte runs, until a 4-byte scalar peel was put in front.

Enums§

Isa
Which instruction set the scanners will actually use.

Functions§

first_escape
First byte at or after from that needs a JSON escape.
first_escape_scalar
THE ORACLE. One byte at a time, and permanent.
first_non_ws
First byte at or after from that is not JSON whitespace.
first_non_ws_scalar
THE ORACLE. One byte at a time, obviously correct, and permanent.
is_ws
Is this byte JSON insignificant whitespace?
isa
The ISA in force, honouring the RJT_ISA override.
isa_available
The widest ISA this build can actually reach on this machine.
needs_escape
Does this byte need a JSON escape?

Type Aliases§

Scanner
A scanner kernel: index of the first interesting byte at or after from.