# Evaluation process protocol
`openlatch evaluate` loads caller-supplied bundles and evaluates caller-supplied
events over stdin/stdout. The client owns verdict semantics. A consumer needs
only a usable binary and its inputs; it does not need enrollment, a daemon,
configuration, credentials, a client checkout, or another repository's release
notification. Invocation is on demand, normally one temporary process per job
stage with bounded concurrency. This protocol imposes no platform startup check.
The implementation is [evaluate.rs](../src/cli/commands/evaluate.rs), backed by
the pure [zone evaluator](../src/zone_eval/mod.rs). This document is the tracked
invocation contract. Prefer additive changes; incompatible framing requires a
deliberate contract change. Client publication owns evaluator readiness.
## Framing and orchestration
- UTF-8 NDJSON: one compact JSON object per line, terminated by `\n`.
The producer serializes a complete object before writing and flushes each line.
Consumers parse JSON objects; object key order is not an acceptance condition.
- Stdout contains protocol frames only. Stderr carries human diagnostics;
never parse stderr or an error's prose to determine a verdict.
- The producer writes its handshake before reading input, then processes frames
in input order. **Feed stdin and drain stdout concurrently**, and drain stderr
too if piped. Writing all input before reading output can deadlock on full pipes.
- `type`, error codes, and skipped-item reasons are open strings. Ignore unknown
optional fields and unknown frame types. An unknown frame does not occupy an
acknowledgment or response position. It cannot replace a required result.
- Blank input lines are ignored. Invalid JSON or non-object input is reported on
stderr and skipped without a reply. A missing, non-string, or unknown input
`type` is ignored without a reply. Known `bundle` and `row` frames receive the
replies specified below; malformed rows receive a row error.
- Track every submitted bundle header and row. Require its corresponding reply,
validate mandatory fields and echoed row ids, and reject missing, extra,
duplicate, malformed or truncated required results. Reconcile replies and
process exit before accepting a run. Exit 0 alone does not prove completeness:
unreadable stdin can terminate the stream and pipe failures can lose output.
Report an unusable contract as a job error and clean up the child process;
never turn missing results into an allow decision.
## Handshake and standalone diagnostics
The first output frame has this shape (version values below are illustrative):
```json
{"type":"handshake","engine_version":"1.0.0","client_version":"0.3.0","schemas_version":"2.0.0-rc.2","protocol_version":1,"corpus_digest":"sha256:…"}
```
`type: "handshake"` is required operational framing. The current producer always
emits all five metadata fields; **consumers treat them as optional diagnostics**,
not compatibility gates or engine pins:
| `engine_version` | String; evaluator semantic version, maintained independently of the client release |
| `client_version` | String; version from `Cargo.toml` |
| `schemas_version` | String; schemas release manifest version |
| `protocol_version` | Integer; frame protocol metadata |
| `corpus_digest` | String; digest of the corpus embedded at build time |
Record available provenance when useful. Do not require equality with another
repository's constants or fixture metadata, or infer conformance from a digest.
Actual operational frame contents determine whether a job can run. Schema
package distribution and its version constraints are a separate contract.
`openlatch evaluate --describe` writes exactly the handshake line and exits 0,
without reading stdin or causing side effects. It is a standalone diagnostic;
no consumer startup invocation is required. `--listen <SOCKET>` remains reserved
and unimplemented: it prints a refusal on stderr, emits no handshake, and exits
2. It opens no socket and does not provide an HTTP or remote decision service.
`--describe` and `--listen` conflict at argument parsing.
## Bundle header and load check
Input:
```json
{"type":"bundle","bundle":{}}
```
Here `bundle` must be the complete inline bundle document, not a filename, URL,
or `bundle_ref`. The empty object above illustrates framing and will be rejected
by bundle loading. Send a header followed by zero rows for a load check; there
is no separate `load_check` command. Each header replaces the active bundle.
Exactly one reply is emitted:
```json
{"type":"bundle_ack","bundle_digest":"sha256:…","skipped":[]}
{"type":"bundle_error","error":{"code":"OL-1241","message":"…"}}
```
| `bundle_ack` | `type`, string `bundle_digest`, array `skipped` |
| `bundle_error` | `type`, object `error` containing string `code` and string `message` |
`bundle_digest` is `sha256:` plus the lowercase hexadecimal SHA-256 of the full
bundle document in RFC 8785 canonical form. It identifies evaluation inputs,
independently of optional engine provenance.
Loading has two stages. A stage-one envelope rejection loads nothing and clears
any previous header bundle. Subsequent rows fail with no bundle until another
header succeeds or a row supplies its own bundle. A stage-two artifact failure
skips that artifact and loads the rest. The acknowledgment must retain every
skipped artifact; an author must be able to see which rules did not activate.
An empty `skipped` array is required when none were skipped.
Each skipped item has three mandatory string fields:
```json
{"kind":"t1_predicate_tree","id":"artifact-id","reason":"body_parse_error"}
```
`kind` and `id` are empty strings when absent from the artifact. Current reasons
are `unknown_kind`, `body_parse_error`, `bad_pattern`, `below_floor`, and
`float_present`; consumers must also retain unfamiliar reasons. Bundle floors
and body validation remain engine-owned load rules, unchanged by optional
handshake provenance.
## Request row and caller-owned state
```json
{"type":"row","id":"event-1","event":{},"state_in":null,"now_ms":1757000000000,"bundle":null}
```
| `type` | Required literal `"row"` |
| `id` | Caller-chosen JSON correlation value, echoed verbatim; omission yields `null`. Callers should supply unique ids within a run. |
| `event` | Required object deserializable as the engine's `Event`; unknown fields are ignored. |
| `now_ms` | Required signed 64-bit integer milliseconds since Unix epoch, injected by the caller; no wall-clock default or float coercion. |
| `state_in` | Session state object or `null`; omission has the same meaning as `null`. |
| `bundle` | Optional inline bundle override for this row only. Omission or `null` uses the header. |
The event shape is defined by [Event](../src/zone_eval/types.rs): string fields
`event_type`, `session_id`, `tool_use_id`, `tool_name`; raw JSON `tool_input` and
`binding`; optional `tool_result`, `agent`, `env`, `session`, and `spend_delta`.
All event fields default when omitted, so `{}` is a valid event. Supply the
stored event's available context rather than filling missing facts by guessing.
An unknown event type is valid. `env` paths are caller-supplied strings used for
classification, never instructions to inspect the host filesystem.
The session state shape is:
```json
{"c":[],"f":[],"t":[],"a":[],"run":null}
```
`c`, `t`, and `a` contain signed 64-bit integers; `f` contains booleans.
`run` is `null` or `{"shape":null,"len":0}`, where `shape` may also be a string
and `len` is a signed 64-bit integer. Omitted arrays default to empty and omitted
`run` to `null`. Array lengths and presence of `run` must match the active
bundle's computed state layout; a mismatch is rejected, never padded.
**Null is unavailable state, not a zeroed register file.** It invokes each
Tier 2 artifact's `on_evict` behavior. An explicitly zeroed object matching the
layout means registers are available. Preserve this distinction.
The process keeps no session history. The caller owns independent state for
each session and evaluation stage. Replay events in stored event-time order
with a stable tie-breaker and pass their stored timestamps as `now_ms`. For a
given session, wait for a successful `state_out` before submitting its next
dependent row and use that output as the next `state_in`. Independent sessions
may interleave but must never share registers. A row error advances no state;
the next row receives the last successful state. Switching bundles or using an
override still requires state appropriate to that bundle's layout.
An override does not change the header or emit an extra acknowledgment. A
rejected override fails only that row and leaves the header intact. To inspect
skipped artifacts in an override, load-check that document with a bundle header
(for example in a separate load-check stage); row responses do not carry
`skipped`. A row may supply its own valid bundle even without a header.
## Row responses
Exactly one response per recognized input row, in input order:
```json
{"type":"response","id":"event-1","engine_version":"1.0.0","bundle_digest":"sha256:…","decision":{},"state_out":{"c":[],"f":[],"t":[],"a":[],"run":null}}
{"type":"response","id":"event-2","engine_version":"1.0.0","bundle_digest":"sha256:…","error":{"code":"OL-1240","message":"…"}}
```
`decision: {}` above is a framing placeholder. A success carries the complete
frozen engine `Decision` object and a session state object. Consumers preserve
the decision opaquely, including unfamiliar fields; they must not redefine
verdict semantics or silently discard required output.
Every response requires `type: "response"`, echoed `id`, and `bundle_digest`
(string when a bundle was selected, `null` when none loaded). A success requires
both `decision` and `state_out` and no `error`. A row failure requires `error`
with string `code` and `message`, and carries **neither `decision` nor
`state_out`**. `engine_version` is optional diagnostic provenance to consumers;
the producer currently emits it on both shapes.
Current row errors: `OL-1241` rejected inline bundle; `OL-1240` malformed event,
timestamp or state; `OL-1242` no bundle; `OL-1243` state-layout mismatch;
`OL-1244` evaluation panicked. Codes remain open strings. Row failures are
reported and processing continues, including a success after an error.
## Exit codes and purity
| 0 | No row errors counted; also a successful `--describe` or zero-row load-check run |
| 1 | At least one row error response; other rows may have succeeded |
| 2 | Could not start, including reserved `--listen` or argument usage errors |
A `bundle_error` alone does not cause exit 1. Inspect the load-check frame even
when the process exits 0. Unexpected process termination, broken transport, or
incomplete required output is a job failure, separate from a reported row error.
These meanings are specific to evaluation, not the CLI's health-report codes.
Evaluation performs no network calls, enrollment, configuration or keyring
reads, audit writes, outbox spooling, deduplication, clock reads, or randomness.
It accesses no filesystem paths driven by an event or bundle. Inputs arrive
through stdin and results leave through stdout; diagnostics use stderr. Caches
may hold data derived from supplied inputs, never session history. Live daemon
and hook enforcement and platform ingest behavior are unaffected.
## Producer fixture and verification
[evaluate-golden-transcript.json](../tests/fixtures/evaluate-golden-transcript.json)
contains actual `--describe` and stream output, a nonempty skipped-artifact
acknowledgment, a success, a row error, and a later success threaded from the
last successful state. [evaluate_transcript.rs](../tests/evaluate_transcript.rs)
regenerates it from the real binary and fails on any producer drift:
```bash
cargo test --test evaluate_transcript
# After reviewing an intentional producer change:
OL_UPDATE_TRANSCRIPT=1 cargo test --test evaluate_transcript
```
Never hand-author expected stdout. The platform vendors a copy from this
tracked client fixture with source provenance. It tests operational compatibility
against that real transcript without requiring synchronized source commits or
exact engine, client, schema, protocol, or corpus metadata equality. The client
retains the strict producer regeneration check.
The semantic corpus remains client-owned. Run the full module check with
`cargo test --features policy --test zone_eval_conformance -- --include-ignored --nocapture`;
report evaluated rows and errors separately. The
[batch differential runner](../tools/zone-eval-ref/README.md#the-batch-arm) drives
the real `evaluate` front door, and
[evaluate_side_effects.rs](../tests/evaluate_side_effects.rs) verifies isolation
and sentinel state. A parseable transcript is evidence of framing, not proof
that an evaluator has passed conformance or is ready for publication.