Skip to main content

Module token

Module token 

Source

Enums§

TokenKind

Constants§

AD_JID
BINARY_8
BINARY_20
BINARY_32
DICTIONARY_0
DICTIONARY_1
DICTIONARY_2
DICTIONARY_3
DICT_VERSION
FB_JID
HEX_8
INTEROP_JID
JID_PAIR
LIST_8
LIST_16
LIST_EMPTY
NIBBLE_8
PACKED_MAX
SINGLE_BYTE_MAX

Functions§

get_double_token
get_single_token
index_of_token
One open-addressing probe over the build-generated table (layout and rationale in build.rs). Keys of length <= 4 — the bulk of probe traffic — are matched exactly by their padded u32; longer keys match a two-load signature first and byte-verify only on signature hits, so a miss of any shape costs a handful of instructions and no full compare.