# Positional lexical postings
The ordinary term-frequency map and non-positional `HMS-TXN-2`/`HMS-TXN-3`
records retain their existing representation. Positions are an opt-in chunk
extension, enabled by `IngestOptions::positions`. They are not reconstructed
from term frequencies when reading an older store. Phrase queries require
positions on every live chunk; otherwise the lexical API returns an explicit
unsupported-index error. Ordinary text/hybrid queries continue to work.
In a binary document record, chunk flag bit 6 indicates a positional payload,
following the term frequencies:
```
byte(version) varint(offset_count) varint(offset){offset_count} bytes(data)
```
Version 1 represents ordinary positional chunks. Version 2 represents a whole
lexical article, selected by `IngestOptions::whole_document`. Both use the same
position encoding. A whole article requires one chunk, lexical encoding, no
embedding, byte offset zero and at most 8 MiB. Ordinary chunks retain their
64 KiB limit. Unknown versions are refused as newer than supported. Readers
predating this extension reject the previously reserved bit rather than reading
the payload as another record.
The offsets number one more than the term count, start at zero and end at the
position byte length. Each term's byte range contains unsigned LEB128 token
position gaps. The first gap is an absolute zero-based position and may be zero;
subsequent gaps are positive. Terms follow the document record's term order.
Positions use the same analyzer as frequencies, before title-frequency weighting.
Each token position occurs once across all terms, and the positions cover the
complete token sequence without holes. Readers check every offset, length,
integer overflow, term frequency and allocation bound before accepting a record.
JSON records place the positional record alongside the legacy frequency entries:
```json
{"$hms_positions":{"version":1,"offsets":[0,1,3],"data":[2,0,1]},"fox":1,"red":2}
```
This encodes `red red fox`. The reserved key cannot be emitted by the alphanumeric
analyzer. It is absent from non-positional records. Chunk text may be discarded;
positions persist in both journal forms, through replay and compaction. The golden
codec fixtures in `tests/golden/lexical-postings-v1/` are regenerated byte-for-byte
by unit tests, including old records and every truncated prefix.
The 128-entry bitpacked posting blocks, their skip endpoints, maximum frequencies
and conservative minimum lengths are derived memory state. Journal replay rebuilds
them from live term tables. No old posting bytes are reinterpreted as blocks and
no persisted block cache is added by this change.