# areev-core
OMS grain model + the `.mg` binary format: canonical serialization, content
addressing, the 12 grain types, and tool-schema rendering. Storage-agnostic —
depends on no other workspace crate; everything else depends on it. OMS
conformance is the compatibility contract with other implementations.
## The `.mg` blob
`blob = 9-byte header ++ canonical MessagePack payload`.
Content address = `SHA-256(entire blob, header included)` → `Hash([u8;32])`.
Header (`src/format/header.rs`): `version(0x01) | flags | grain_type_byte |
ns_hash(u16 BE) | created_at_sec(u32 BE)`. `ns_hash` = first 2 bytes of
SHA-256(namespace). Flags bits: signed 0x01, encrypted 0x02, compressed 0x04,
has_content_refs 0x08, has_embedding_refs 0x10, ai_generated 0x20, bits 6–7
sensitivity. Byte-exact header vector pinned in `header.rs` tests
(`01 00 01 a4 d2 …`).
## Canonical serialization invariants — DO NOT BREAK
Any change here silently changes content addresses of every grain ever written
and breaks OMS §21 conformance. Treat as frozen unless the spec moves:
- **NFC-normalize every string before hashing** (`serialize.rs` `nfc_string`).
Unicode composition variants deliberately collapse to one content hash.
- **Sorted map keys** — maps are built as `BTreeMap`, emitted in sorted order.
- **Compact keys mandatory** — writers MUST emit the short forms in
`field_map.rs`. Exceptions that stay uncompacted: `content`, `nodes`,
`edges`, `trigger`, `retries`. Tool's content compacts to `cnt` to avoid
colliding with Event `content`.
- **Omit-when-default** — `None`/empty fields omitted; `ToolKind::Execution`
and `ExecutorKind::Axtion` omitted, to keep legacy blobs byte-identical.
- Timestamps: epoch **ms** in the payload, epoch **sec** in the header.
- Authenticity never touches the blob: an attestation is a separate grain in
`agent:attest` (`authz::ATTEST_NS`, built and verified in
`areev-store::attest`). Header bit 0 (`is_signed`) is reserved for OMS §9
and never set; a `0x84` COSE prefix is refused by `deserialize_blob`.
## Module map
- `error.rs` — `AreevError`, `Result<T>`, `Hash` newtype (hex display/serde).
- `time.rs` — `iso8601_to_ms` + `now_ms`. The ONE datetime parser (invariant 6:
no datetime dependency). It lives here rather than in the crate that first
needed it (`areev-store`'s importers, which now re-export it) because the
credential map's `expires_at` needs the same parse and `authz` sits below the
store. The grain path deliberately does not call `now_ms` — serializers and
`audit_observation` take `now_ms` as a parameter so content addresses stay
reproducible in tests.
- `format/header.rs` — `MgHeader`, flag bits, `content_address()`.
- `format/field_map.rs` — long↔short key tables.
- `format/serialize.rs` — `serialize_grain()` → `(blob, Hash)`.
- `format/deserialize.rs` — `deserialize_blob()` → `DeserializedGrain`; typed
reconstructors (`to_fact`/`to_event`/…), `embedding_text()`, `base_text()`.
- `format/tool_schema/` — render Tool-definition grains to 9 provider formats.
- `types/grain.rs` — `Grain` trait, `GrainCommon`, `GrainType`, `GrainData`.
- `types/registry.rs` — `GRAIN_TYPES` table: **source of truth** for
byte/name/plural/add_via_set/queryable per type.
- `types/capability.rs` — the Tool grain's `capabilities` vocabulary (#101) and
the `AllowedHost` URL-prefix grammar it shares with the outbound allowlist.
Lives HERE, not in `areev-run`, because three layers at different heights
must agree on what a declaration means: CAL's write validation, the run
manifest's start-time freeze, and the broker's per-call enforcement — and
`areev-cal` sits below `areev-run`. Two parsers is how a tool becomes
writable and then unrunnable.
- `types/<type>.rs` — the 12 grain structs: Fact 0x01, Event 0x02, State 0x03,
Workflow 0x04, Tool 0x05, Observation 0x06, Goal 0x07, Reasoning 0x08,
Consensus 0x09, Consent 0x0A, Skill 0x0B,
Recommendation 0x0C.
## Adding / changing a grain type
A registry row is necessary but not sufficient. Serialization dispatches on
`downcast_ref` chains, so you must also touch:
1. `types/registry.rs` — the metadata row (a test forces coverage of all types).
2. `format/serialize.rs` — `add_type_specific_fields` downcast arm.
3. `format/deserialize.rs` — reconstruction arm.
4. `field_map.rs` — compact keys for new fields (collision test exists).
## tool_schema/
9 `ProviderKind`s: openai-tools, openai-responses, anthropic-tools,
gemini-tools, mcp-tools (JSON) + hermes, llama31, markdown-tools, sml-tools
(text). Adding a format: new variant + `parse`/`as_str`/`ALL`/`is_text`, an
adapter file with `render(&Tool)`, wired into `render_json`/`render_text`.
Outputs must be deterministic. Text adapters must sanitize via `escape.rs`.
`parse.rs` is the inverse for 5 formats (ReDoS-guarded regexes).
## Tests
`cargo test -p areev-core`. `tests/oms_conformance.rs` = OMS §21 vectors
(Vector 1 byte-exact, content address `3288d0d4…` reproduced) + roundtrip
determinism per type. Grains are built inline — no fixture files. Heavy inline
`#[cfg(test)]` suites in serialize/deserialize/field_map/registry.
## Gotchas
- There is no signing code in this crate, on purpose. The former COSE
scaffold (`serialize_grain_signed`, a `signing` feature) was retired by the
§10 decision "Authenticity is an attestation grain, not an envelope"
(`docs/grain-attestation-plan.md`): signing lives in `areev-store::attest`
as a detached grain, so the format and every content address stay frozen.
Do not reintroduce an envelope here without an OMS §9 decision.
- Deserialize is forward-compatible: unknown enum wire strings are ignored,
not errors.
- `base_text()` (reranker input) ≠ `embedding_text()` (embedder input) —
deliberately decoupled, pinned by tests.
- Skill has no `proficiency` field — it aliases `common.confidence`.
- Naming quirk: the Tool grain is called "action"/"axtion"
in older comments and field names (`axtion_uri`).