Module common 

Core model-independent logic: state serialization (Python json.dumps parity), option rendering, token-sequence construction, collation, and calibration helpers.

Ported from laya/common.py (source of truth) and mirrored against laya-ts/src/common.ts.

Structs

CollatedBatch

A padded batch ready to be turned into tensors.

InternalQ

A validated + normalized question, ready for sequence building and decoding. crit is kept as a JSON value in the shape the type expects (choice: object, score: array, noul: object-or-null), matching the normalization in Agent::to_internal.

Item

One tokenized (state, question) pair before collation.

Enums

QType

The three question types. Numeric indices match Python QTYPES (choice=0, score=1, noul=2) — they select the model’s type embedding row.

Constants

TEMP_MAX

TEMP_MIN

Traits

Tokenizer

Abstraction over a checkpoint tokenizer. Mirrors TokenizerLike in laya-ts. encode corresponds to HF tokenizer(text, add_special_tokens=False)["input_ids"].

Functions

answer_confidence

Probability mass on the answer being reported: max(p[:k]), clamped to [0, 1].

build_sequence

Build one token sequence + option marker positions.

clamp_temperature

A usable temperature: t confined to [TEMP_MIN, TEMP_MAX], falling back to 1.0 if not finite.

collate_items

Collate groups of items into one padded batch. groups is a list of per-state item lists, flattened in order (mirrors collate_items in laya/common.py). Returns None if empty.

confidence_from_probs

Normalized Shannon-entropy confidence: 1 - H(p)/log(k), clamped to [0, 1]. p is expected to already hold the k option probabilities.

drop_stale_option_order

Remove an option_order a narrowed choice question can no longer honour, warning once per qid. Returns true if an order was dropped. Mirrors upstream shortlist._drop_stale_order.

jev_confidence

Jev’s confidence for a choice/score answer’s probabilities, or None (upstream #302).

py_json

Replica of Python json.dumps(value, ensure_ascii=False, separators=(", ", ": ")).

render_criterion

Render one criterion value as text (strings pass through, structured values become JSON).

render_options

Render option texts in label-index order. Noul semantic order is always [false, true].

resolve_noul_labels

Resolve noul labels to (false_label, true_label), validating the mapping.

round4

Round to 4 decimals like Python round(x, 4) (the precision Laya publishes).

round_half_even

Round to ndigits decimals using round-half-to-even, matching Python’s built-in round.

serialize_state

A state as the model sees it: raw strings pass through, everything else becomes compact JSON.

softmax

Numerically stable softmax.

temp_bucket

Temperature bucket key, e.g. "choice:3-5".

unpermute_probs

Put a slot-ordered probability row back into the caller’s option order.