# Serialization Strategy
Tokitai uses two different JSON policies for paper-critical artifacts:
- Canonical renderers produce compact, deterministic JSON bytes for trace
persistence, artifact digests, and manifest signatures.
- Permissive importers accept ordinary JSON object field ordering for version-1
trace artifacts while preserving strict schema validation.
This split is intentional. Digest and signature stability depend on stable byte
rendering, but reviewer-facing artifact interchange should not depend on object
field order.
## Canonical Renderers
Canonical renderers are the `to_json` implementations and public wrapper
functions in `src/verify/mod.rs`, including:
- `proof_replay_json_report`
- `proof_replay_trace_manifest_json`
- `proof_assistant_adapter_registry_json_for_capabilities`
- `proof_assistant_checker_transcript_json`
- `proof_assistant_checker_output_json`
- `proof_assistant_native_admission_json`
- `checker_result_index_json`
- `proof_replay_trace_tuning_json`
- `proof_replay_trace_benchmark_json`
The canonical renderer contract is:
- field names and order are stable for artifact version `1`;
- strings use Tokitai's deterministic JSON escaping;
- optional fields render as either a quoted string or `null`;
- arrays preserve semantic order, such as manifest artifact order and theorem
order;
- `proof_replay_artifact_digest` and trace signature payloads are computed from
canonical bytes, not from the imported source text.
## Permissive Imports
P118 and P119 route paper-critical version-1 JSON imports through a small
dependency-free object lookup layer. These importers accept reordered object
fields, pretty-printed whitespace, and nested reordered objects for the trace
artifact chain.
They still reject:
- missing required fields;
- unknown fields;
- duplicate fields;
- unsupported artifact versions;
- malformed strings, arrays, or objects;
- invalid digests, theorem counts, signatures, registry bindings, native
admissions, or checker result fingerprints.
After import, artifacts normalize back to canonical bytes by calling their
canonical renderer. This makes field-order-independent import compatible with
stable digests and signatures.
## Serde Decision For P120
P120 keeps the dependency-free parser layer for the paper artifact path instead
of adding serde-backed serialization immediately.
Rationale:
- The current artifact set already has generated schema bundles, Rust schema
constants, field-order-independent imports, and tamper tests.
- Adding serde now would not change the research contribution; it would mostly
replace already-guarded parsing mechanics.
- Canonical byte rendering is a paper-critical invariant. A serde migration must
prove that canonical JSON bytes and artifact digests remain unchanged.
- Avoiding a new dependency keeps the reproducibility package smaller for
reviewers.
Serde remains acceptable for future internal models if it is kept behind the
canonical renderer boundary. A future serde migration should add tests proving
that serde-derived values normalize to the same canonical JSON and digest bytes.
The post-submission serde migration plan is tracked in
`docs/paper/post_submission_serialization.md`. That plan preserves the current
canonical renderers and schema guards as authoritative for P132, and requires
byte-equivalence and digest-equivalence tests before serde-backed paths can be
trusted for broader audit artifacts.
The current canonical renderers and schema guards as authoritative for P132.
## Compatibility Invariants
For every version-1 paper artifact:
- `parse(canonical_json(x)) == x`;
- `canonical_json(parse(reordered_json(x))) == canonical_json(x)`;
- `digest(canonical_json(parse(reordered_json(x)))) == digest(canonical_json(x))`;
- unsupported versions and schema drift remain validation errors.
These invariants are guarded by `tests/json_parser_regression.rs`,
`tests/schema_guards.rs`, and the trace tamper tests in
`tests/audit_traces_example.rs`.