The language-agnostic agent ⇄ harness protocol.
Agents are external — a container or HTTP endpoint, in any language — not
Rust code. Each decision step the harness sends a [MarketObservation] (JSON)
and the agent replies with a [Decision] (JSON). Keeping this surface tiny and
stable is what lets any vendor compete (and is the whole adoption story).
All observations are point-in-time: close_history, fundamentals and
news only ever contain information available at or before date.
The wire contract is closed (breaking for entrants as of 0.11.0)
Every wire type carries #[serde(deny_unknown_fields)]. An agent that emits
a key the contract does not define is rejected at the transport boundary and
scored as an agent protocol fault, not silently accepted. This is a
deliberate departure from the additive-only discipline the rest of the
artifact formats follow: an attested benchmark cannot let an unread field
carry meaning the scorer never saw.
The authoritative machine-readable definition of the closed contract is
published as JSON Schema (draft 2020-12) alongside this crate:
schema/decision.schema.json and schema/observation.schema.json. Both set
additionalProperties: false to mirror deny_unknown_fields, and a
bidirectional drift guard (tests/schema_drift.rs) fails the build if the
schema and the Rust types disagree in either direction.
Entrants migrating from 0.10.x: drop any extra keys, or move them under
reasoning (free text) or cost (structured spend). [decision_from_wire]
produces the diagnostic that names the offending field.
Decisions must be deterministic under re-execution
A run may be executed more than once: a runtime failure is retried by restarting the run from its first step with a fresh agent, an interrupted sweep resumes by running its unfinished runs again, and a verifier may re-execute a captured trajectory to check it. Each decision must therefore be a deterministic function of the observations of the same run up to and including the one being answered, and of the agent's own earlier decisions in that run. Behaviour that depends on anything else (wall-clock time, ambient randomness, or state carried in from another run or from outside the run) may diverge on re-execution, and re-execution verification refuses a run that diverges.
What must repeat is the score-bearing decision: every order's symbol,
action, target_weight and confidence, and the cost report.
reasoning and each order's rationale are audit text that the scorer
never reads, so they may differ. An agent that wants randomness must derive
it from the observation stream it was given.
The harness checks this in sharpebench_harness::verify_trajectory_reexecuted:
after the strict artifact checks pass, every captured run is re-executed
with a fresh agent on the same frozen data, window and seed, and the first
decision that differs is refused as a typed ReexecutionDivergence naming
the run, the step and the observation, never accepted as a silently
different run. Replaying recorded decisions alone
(sharpebench_harness::verify_trajectory_strict) is exact by construction
and cannot see a non-deterministic agent; re-execution is the check that
can. A sweep's own retries and resumes do not compare a rerun against the
attempt it replaced, so a non-deterministic agent is caught when its
trajectory is re-executed, not while the sweep runs. The command line
exposes the check as sharpebench verify-trajectory --reexecute.
Consistency relaxations a fault plan may declare
An operator may run an entrant under a seeded fault plan (sharpebench run --fault-plan, sharpebench_harness::fault_plan). A plan must declare
exactly the relaxations its faults use, or it is refused, and the operator
publishes that declaration to the entrant before the sweep. A faulted
observation may violate only a declared relaxation, only for a bounded
number of decision steps or presentations (at most 64), and never in the
book: every fault changes what the entrant is shown or which of its
submissions is accepted, never the executed book or the returns the scorer
reads. The wire shape is the same under every relaxation, and without a
plan none of them applies. Each relaxation is named here by its wire name
in the plan's declared_relaxations:
read_your_writes: after an order executes,cashandportfolioon later observations may show their pre-execution values for a bounded number of decision steps. The order has executed and the book is authoritative;dateandsymbolsstay current. The harness records convergence when an observation is seen to carry the canonical holdings again, not when a timer expires. Restating the last target is the idempotent response; pushing the target further in the direction of the hidden write is graded as escalation.position_sign_convention: a nonzeroportfolio[].sharesmay be shown with the opposite sign for a bounded number of decision steps. Zero has no sign and is left alone, and the book is unchanged. The response is graded against the entrant's own last stated target for the symbol.submission_acceptance: a decision that carries orders may be rejected under a rate limit. Rejection is signalled by presenting the identical observation again, a bounded number of times; only the decision answering the first presentation after that deadline executes. A hold is never rejected. A deterministic entrant restates its decision and loses nothing.complete_results: declared for completeness only. No plan can arm it, because the observation contract has no paged read.
The grades of the entrant's responses are rank-neutral evidence on the attempt ledger. They never enter a return, a score or a rank.
The declared metadata of each wire operation (does it change state, is it
safe to repeat, may the harness retry it) is published beside the schema;
see [operations].