Module utf16 

The UTF-8 ⇄ UTF-16 boundary for JS string indices.

A JS String is a sequence of UTF-16 code units, and every index-bearing String.prototype operation counts in those units: length, charAt, charCodeAt, codePointAt, at, indexOf/lastIndexOf, slice, substring, substr, split, padStart/padEnd, s[i], and a RegExp’s .index/lastIndex.

Indices are not the only place the unit sequence is observable. Relational comparison (< <= > >=, 7.2.13 IsLessThan) and the default Array.prototype .sort comparator order strings by code unit too, which is a different order than Rust’s str: Ord (code point / UTF-8 byte order) for any pair that straddles U+E000: a surrogate is 0xD800..0xE000, so every astral character sorts BELOW every BMP character from U+E000 up. See cmp_units.

node-js stores a JS string as a Rust String (UTF-8): fusevm::Value::Str and the host heap’s JsObj::Str are both String, and fusevm is a pinned external dependency, so the storage type is not ours to change. Every JS-visible index therefore has to be translated, and this module is the one place that translation happens. Indices are code-unit counts (U16Index); Rust’s own string offsets are byte counts; the two are only equal on ASCII, and the newtype exists so a function that has both in scope cannot silently pass one where the other belongs.

The lone-surrogate boundary

Rust String cannot hold an unpaired surrogate — char excludes U+D800..=U+DFFF — so an operation that cuts a surrogate pair in half cannot reproduce node’s result exactly. "𝒳".charAt(0) is the lone surrogate \ud835 in node; here it is U+FFFD. This is deliberately the narrowest possible gap:

• Index arithmetic is exact — a cut at a surrogate boundary still happens at the right place, still yields a 1-unit string, and every surrounding index still lines up. "𝒳".length is 2 and "𝒳".charCodeAt(0) is 55349, read from the intact original.
• Printing is byte-identical: node itself writes ef bf bd (U+FFFD) when a lone surrogate reaches stdout, verified with node -e 'process.stdout.write("𝒳".charAt(0))' | xxd.
• Only re-inspecting an extracted half differs — "𝒳".charAt(0).charCodeAt(0) (65533 here, 55349 in node), JSON.stringify("𝒳".charAt(0)), and re-joining two halves back into the original astral character.

Closing that last gap means replacing String with a WTF-8 buffer throughout fusevm and all 47 stdlib modules, which the pinned dependency forbids.

Structs

U16Index

An index into a JS string, counted in UTF-16 code units.

Units

A JS string decoded into its UTF-16 code units, so that index arithmetic can be done directly on the units JS counts.

Functions

byte_of_index

The UTF-8 byte offset corresponding to a UTF-16 index into s. An index that falls inside a surrogate pair rounds down to the start of that code point, so the result is always a valid str boundary.

cmp_units

Lexicographic order over UTF-16 code units — the order JS’s </<=/>/>= and the default sort comparator use (7.2.13 IsLessThan step 3.d compares “the code unit at index k”).

index_of_byte

The UTF-16 index corresponding to a UTF-8 byte offset into s.

is_js_whitespace

ECMA-262 WhiteSpace (11.2) + LineTerminator (11.3): the exact character set String.prototype.trim, ToNumber(string), parseInt and parseFloat skip.

js_trim

s with leading and trailing JS whitespace removed.

js_trim_end

s with trailing JS whitespace removed.

js_trim_start

s with leading JS whitespace removed.

len

The length of s in UTF-16 code units — the value of s.length in JS.

to_string_lossy

Decode UTF-16 code units back to a Rust String, mapping any unpaired surrogate to U+FFFD — the same replacement node performs when a lone surrogate is written to stdout.

to_uint16

ToUint16(n) — the modulo-2^16 wrap String.fromCharCode applies to each argument, so fromCharCode(0x1D4B3) produces U+D4B3 and not U+1D4B3.