# Agent Trace Infra HTTP API
Standalone **Trace Infra Server** — diagnostic trace event storage for Agent Runtime.
| Binary | `agent-trace-infra` |
| Default bind | `127.0.0.1:5140` |
| Audience / service name | `agent-trace-infra` |
| OpenAPI-style prefix | `/internal/v1/trace-infra` |
This server provides **idempotent, chronologically ordered** trace event storage. Trace events are diagnostic records of tool calls, guard-rail evaluations, and lifecycle transitions inside a single run. Unlike conversation messages, trace events are **never replayed into a model prompt**.
---
## Authentication
| **Open** | `AGENT_TRACE_INFRA_SERVICE_TOKEN` unset / empty | All routes accept requests without `Authorization` |
| **Token** | env set to a non-empty string | Internal routes require `Authorization: Bearer <token>` |
- `GET /health` is always unauthenticated.
- Wrong / missing token → `401` / `403` with body `{ "status": <code>, "message": "..." }`.
```bash
export AGENT_TRACE_INFRA_SERVICE_TOKEN='<development-token>'
curl -H "Authorization: Bearer <development-token>" http://127.0.0.1:5140/internal/v1/trace-infra/events/batch \
-H 'content-type: application/json' \
-d '{"events":[{"id":"e1","scope":{"conversationId":"c1","runId":"r1"},"eventType":"tool_call","payload":{}}]}'
```
---
## Error shape
All non-2xx JSON errors:
```json
{ "status": 400, "message": "human readable reason" }
```
| 400 | Validation failure: empty id, empty conversationId, etc. |
| 401 | Missing bearer when token mode is on |
| 403 | Bearer present but wrong |
---
## `GET /health`
**Auth:** none
**Response `200`:**
```json
{ "status": "ok", "service": "agent-trace-infra" }
```
---
## `POST /internal/v1/trace-infra/events/batch`
Append trace events. Idempotent: re-sending an event with the same `id` within the same partition is a no-op (counted in `skipped`).
**Auth:** token mode if configured
### Request body
```json
{
"events": [
{
"id": "e1",
"scope": { "conversationId": "c1", "runId": "r1" },
"eventType": "tool_call",
"payload": { "tool": "search", "args": {} },
"sequence": 0,
"createdAtMs": 0
}
]
}
```
| `events` | `[RuntimeEventRecord]` | yes | |
| `events[].id` | string | yes | Used for idempotent deduplication |
| `events[].scope.conversationId` | string | yes | Partition key (part 1) |
| `events[].scope.runId` | string | no | Partition key (part 2); groups events from one turn |
| `events[].eventType` | string | yes | Coarse kind: `tool_call`, `guard_rail`, `turn_started`, `turn_finished`, etc. |
| `events[].payload` | JSON value | no | Free-form payload; auto-redacted on store |
| `events[].metadata` | JSON value | no | Free-form metadata; auto-redacted on store |
| `events[].sequence` | i64 | no | `0` = auto-assign (monotonic per partition) |
| `events[].createdAtMs` | i64 | no | `0` = auto-fill to current epoch millis |
### Response `200`
```json
{ "appended": 1, "skipped": 0 }
```
| `appended` | usize | Number of events newly stored |
| `skipped` | usize | Number of events omitted because their `id` already existed in the same partition |
### curl
```bash
curl -s -X POST http://127.0.0.1:5140/internal/v1/trace-infra/events/batch \
-H 'content-type: application/json' \
-d '{"events":[{"id":"e1","scope":{"conversationId":"c1","runId":"r1"},"eventType":"tool_call","payload":{"name":"execute_cmd"}}]}'
```
---
## `POST /internal/v1/trace-infra/events/search`
Search for events in a partition. Returns events in chronological order, oldest first.
**Auth:** token mode if configured
### Request body
```json
{
"scope": { "conversationId": "c1", "runId": "r1" },
"eventType": "tool_call",
"limit": 100
}
```
| `scope` | `TraceScope` | yes | At minimum `conversationId`; `runId` isolates a single turn |
| `eventType` | string | no | When set, only events with this exact type are returned |
| `limit` | usize | no | Default `100`, clamped to `1000` |
### Response `200`
```json
{
"items": [
{
"id": "e1",
"scope": { "conversationId": "c1", "runId": "r1" },
"eventType": "tool_call",
"payload": { "name": "execute_cmd" },
"sequence": 1,
"createdAtMs": 1700000000000
}
]
}
```
Records are in **chronological order, oldest first**. The `limit` applies to the tail of the filtered list: you get the last `limit` records.
### curl
```bash
curl -s -X POST http://127.0.0.1:5140/internal/v1/trace-infra/events/search \
-H 'content-type: application/json' \
-d '{"scope":{"conversationId":"c1","runId":"r1"},"limit":100}'
```
---
## Guarantees
### Ordering
All reads return events in **chronological order, oldest first**, sorted by `(created_at_ms, sequence)`. Ties break by `sequence`.
### Idempotency
Append is idempotent by `id`. If an event with the same `id` already exists in the same partition, the new event is silently dropped and counted in `skipped`. This makes retry after a network timeout safe: you will never get duplicate trace events.
### Per-run partitioning
Events are partitioned by `(conversation_id, run_id)`. Events from one run are never mixed with another run's events, even when they share the same conversation. This makes it safe to reconstruct a single execution from its trace without guessing from timestamps.
### Server-side redaction
Every event's `payload` and `metadata` are redacted before storage. Fields whose names look like credentials (`api_key`, `token`, `authorization`, `secret`, `password`, etc.) are replaced with `<redacted>`. This is done server-side because trace data is the most likely to be exported to a log aggregator and retained — a leaked credential there outlives the request that leaked it.
### Bounded retention (per-partition)
Trace is high-volume: a single turn can produce dozens of events. The store keeps at most 10,000 events per partition (configurable via `TRACE_INFRA_MAX_EVENTS_PER_RUN`), evicting the oldest entries when the cap is exceeded. This prevents the store from becoming a slow memory leak.
### Trace events are never replayed into a model prompt
Unlike conversation messages, trace events are diagnostic records. They are never replayed into a model prompt, so losing one is acceptable. This is the fundamental design distinction that justifies a separate service.
---
## Page-size limits
| `DEFAULT_PAGE_SIZE` | 100 |
| `MAX_PAGE_SIZE` | 1000 |
An unbounded read is a denial-of-service vector, so every page-size parameter is capped at `MAX_PAGE_SIZE`. The default is `DEFAULT_PAGE_SIZE` when the request omits the field.
---
## Environment variables
| `AGENT_TRACE_INFRA_SERVICE_TOKEN` | Static bearer token for auth; unset/empty = open mode |
| `TRACE_INFRA_MAX_EVENTS_PER_RUN` | Per-partition retention cap (default 10,000) |
---
## Curl cookbook (local open mode)
```bash
# Health
curl -s http://127.0.0.1:5140/health
# Append an event
curl -s -X POST http://127.0.0.1:5140/internal/v1/trace-infra/events/batch \
-H 'content-type: application/json' \
-d '{"events":[{"id":"e1","scope":{"conversationId":"c1","runId":"r1"},"eventType":"tool_call","payload":{"name":"ls"}}]}'
# Append a second event (different run)
curl -s -X POST http://127.0.0.1:5140/internal/v1/trace-infra/events/batch \
-H 'content-type: application/json' \
-d '{"events":[{"id":"e2","scope":{"conversationId":"c1","runId":"r2"},"eventType":"observation","payload":{"result":"ok"}}]}'
# Search events in a run (chronological, oldest first)
curl -s -X POST http://127.0.0.1:5140/internal/v1/trace-infra/events/search \
-H 'content-type: application/json' \
-d '{"scope":{"conversationId":"c1","runId":"r1"},"limit":100}'
# Filter by event type
curl -s -X POST http://127.0.0.1:5140/internal/v1/trace-infra/events/search \
-H 'content-type: application/json' \
-d '{"scope":{"conversationId":"c1","runId":"r1"},"eventType":"tool_call"}'
```
---
## Related crates
| `agent-trace-contract` | Types + path constants + this API surface |
| `agent-trace-runtime` | In-memory trace store implementation |
| `agent-trace-server` | Axum host + optional static token auth |