Expand description
Labeled-example schema and strict JSONL parse/encode over &[u8].
Implemented by plan 02-03.
Labeled-example schema and strict JSONL parse/encode over &[u8].
Deserialization is deny_unknown_fields at the bytes -> typed boundary: bytes that
arrive from object storage are untrusted, and a silently ignored extra field is how a
schema drift becomes a silent data change.
§Field names and order are load-bearing
[LabeledExample]’s five fields are id, input, label, label_text, source_split, in
that order, matching the field order of the D-06 baseline row struct this type absorbed
(StanceSample, formerly in crates/apr-cli/src/commands/data_tweeteval.rs, removed in
plan 02-06 once the adapter read through this type instead). serde_json emits struct
fields in declaration order, so keeping the order means relocated JSONL output stays
byte-compatible with datasets already produced on developer machines. Reordering
the fields is a manifest schema_version bump, never a silent edit.
§Why the byte round-trip matters
[encode_jsonl] is the canonical re-encoding of accepted rows, and
encode_jsonl(parse_jsonl_bytes(b)?)? == b holds for canonical input. The split
constructors rely on that: one derives its source_hash from the ingested buffer and
the other from the re-encoding of typed rows, and the two must agree or a dataset
fingerprint would depend on which door the caller came through.
Structs§
- Labeled
Example - One labeled example, the atom of every split in this protocol.
Functions§
- encode_
jsonl - Canonically encode rows as JSONL:
serde_json::to_writerper row, thenb'\n'. - parse_
jsonl_ bytes - Parse a JSONL buffer into typed rows, preserving input order.