antares-format 0.1.0

Open Antares format (.ant): zstd-framed NDJSON container for graph + evidence + belief exchange
Documentation

Open Antares format (.ant) — v0.1.

A self-contained, compressed, streamable container for exchanging a selection of an Antares world model: schema types, vertices, edges, observations, evidence (structured AND unstructured together), beliefs, and vector docs (vectors must ride: the server does not persist vector indexes).

Container

One zstd-compressed stream of NDJSON records:

{"kind":"manifest", ...}      exactly one, first line
{"kind":"schema_type", "data":{...}}
{"kind":"vertex",      "data":{...}}
{"kind":"edge",        "data":{...}}
{"kind":"observation", "data":{...}}
{"kind":"evidence",    "data":{...}}
{"kind":"belief",      "data":{...}}
{"kind":"vector",      "data":{...}}
{"kind":"trailer", "counts":{...}, "sha256":"..."}   exactly one, last line
  • data payloads are the serde JSON of the corresponding ant-types records (the same property encoding the wire and store use).

Property values (v0.3)

v0.2 carried property values as bare untagged JSON scalars: null | bool | number | string | object. That set cannot express the SQL types — DECIMAL, DATE, TIME, TIMESTAMP, UUID and BLOB are all JSON strings, so a reader could not tell a date from a string that looks like one, and the type was lost on the first round-trip.

v0.3 keeps those five shapes EXACTLY as they were and adds a tagged envelope for the typed values:

{"$ant":"decimal",   "v":"12345678901234567.89"}  exact, never f64
{"$ant":"date",      "v":"2024-03-01"}
{"$ant":"time",      "v":"12:30:45.123456"}
{"$ant":"timestamp", "v":"2024-03-01T12:00:00+02:00"}   offset kept
{"$ant":"uuid",      "v":"6ba7b810-..."}
{"$ant":"bytes",     "v":"<base64>"}
{"$ant":"int32",     "v":-2147483648}
{"$ant":"int16",     "v":-32768}
{"$ant":"array",     "v":[ <property values> ]}

An object is an envelope ONLY when it has exactly the two keys $ant and v and $ant names a known type; anything else is an ordinary JSON document value. So a producer's own document with a $ant field still round-trips as that document.

This is a MINOR bump because it is additive under the compatibility policy below: a v0.2 reader reads a v0.3 file without error, and any value it already understood is byte-identical. What it loses is the type — an envelope decodes as a plain JSON object rather than as a decimal — which is precisely what "the file is ahead of this reader" is there to signal.

  • The trailer's sha256 is over every preceding UNCOMPRESSED line including newlines (manifest through the last record), so truncation and tampering are detectable without a second pass.
  • File identification: the zstd magic plus a first record with kind == "manifest" and a supported format version.
  • Records of unknown kind are skipped by readers (forward compatibility); additive fields inside data follow serde defaults.

Selection semantics (writer-side contract)

A .ant file carries whatever selection the exporter chose (whole scope, a seed set + traversal, a 50-row digest). The manifest records the selection descriptor verbatim so the consumer knows what the file claims to contain; evidence closure is the exporter's obligation: every evidence_id referenced by an exported observation/edge should have its evidence record included.