Skip to main content

Module schema

Module schema 

Source
Expand description

Labeled-example schema and strict JSONL parse/encode over &[u8].

Implemented by plan 02-03. Labeled-example schema and strict JSONL parse/encode over &[u8].

Deserialization is deny_unknown_fields at the bytes -> typed boundary: bytes that arrive from object storage are untrusted, and a silently ignored extra field is how a schema drift becomes a silent data change.

§Field names and order are load-bearing

[LabeledExample]’s five fields are id, input, label, label_text, source_split, in that order, matching the field order of the D-06 baseline row struct this type absorbed (StanceSample, formerly in crates/apr-cli/src/commands/data_tweeteval.rs, removed in plan 02-06 once the adapter read through this type instead). serde_json emits struct fields in declaration order, so keeping the order means relocated JSONL output stays byte-compatible with datasets already produced on developer machines. Reordering the fields is a manifest schema_version bump, never a silent edit.

§Why the byte round-trip matters

[encode_jsonl] is the canonical re-encoding of accepted rows, and encode_jsonl(parse_jsonl_bytes(b)?)? == b holds for canonical input. The split constructors rely on that: one derives its source_hash from the ingested buffer and the other from the re-encoding of typed rows, and the two must agree or a dataset fingerprint would depend on which door the caller came through.

Structs§

LabeledExample
One labeled example, the atom of every split in this protocol.

Functions§

encode_jsonl
Canonically encode rows as JSONL: serde_json::to_writer per row, then b'\n'.
parse_jsonl_bytes
Parse a JSONL buffer into typed rows, preserving input order.