modelcontextprotocol 0.2.4

Filesystem, Fetch, Memory, Shell, Skills, and Agents MCP servers in a single binary, implementing the Model Context Protocol 2026-07-28 specification.
# OpenRouter E2E acceptance

The gated, real-network acceptance suite in `tests/openrouter_e2e.rs` is the
legacy chat-tool acceptance suite. It does not exercise the Skills or Agents
tools. It drives real MCP servers, the real protocol lifecycle, and a real
frontier model over the real OpenRouter API.

## What the suite does

1. Spawns the filesystem, fetch, memory, and shell servers in a disposable environment and
   connects via the modern `server/discover` lifecycle, asserting the
   negotiated protocol version (`2026-07-28`), server identity
   (`mcp-filesystem`, `mcp-fetch`, `mcp-memory`, `mcp-shell`),
   capabilities, and instructions.
2. Derives every OpenRouter function schema from the runtime MCP `Tool`
   definitions through a single deterministic normalizer/validator
   (`tests/openrouter/schema.rs`) — no duplicated handwritten schemas.
3. Runs one forced `assistant -> tool -> assistant` roundtrip per case:
   the model must call exactly the intended tool with exactly the intended,
   schema-valid arguments; the MCP tool executes; the result returns with
   the matching `tool_call_id`; tools are resent; the final answer must be
   bounded and non-empty. An independent programmatic oracle checks the MCP
   result and fixture side effects (disk content, JSONL state, content
   blocks).
4. Consumes the `fetch` prompt and the `memory://knowledge-graph` resource
   through one bounded real request each, with the content fed into the
   model.
5. Enforces coverage: `assert_coverage` requires exact set equality between
   the case catalog and the live tool inventory, plus per-parameter
   coverage (provided for every parameter; additionally omitted for
   optional parameters). Adding a tool or parameter without updating the
   catalog fails the suite.

Fetch correctness never depends on the public internet: every fetch case
uses a deterministic local `tiny_http` fixture (including robots.txt
modes). Shell cases use an isolated temp cwd and this test binary as a
deterministic helper; no destructive commands are ever executed.

## Model and bounds

- Endpoint: `POST https://openrouter.ai/api/v1/chat/completions`, Bearer
  auth from `OPENROUTER_API_KEY` (environment only).
- Required default model alias: `~deepseek/deepseek-v4-flash-latest`
  (OpenRouter `~` = latest of the family). `OPENROUTER_MODEL` is accepted
  as an override for diagnostics, but the acceptance run must use the
  default alias; the response `model` field reports the concrete resolved
  model.
- Forced tool shape `{"type":"function","function":{...}}`;
  `parallel_tool_calls: false`, `stream: false`, `temperature: 0`.
- Per-request: timeout 45 s, response body ≤ 1 MiB, `max_tokens` 512 per
  tool call and 200 per final answer/consumption request.
- Retry policy: at most 2 attempts per request, retrying only transport
  errors, 429s, 5xx, and `Retry-After` responses (backoff capped at 5 s).
  In addition, a whole roundtrip is retried up to 2 times for
  model-generation flakes only: a model-echo deviation (dropped/rewritten
  argument — the argument guard runs again and still blocks before any MCP
  execution), truncated (unparseable JSON) arguments, a `length` finish
  reason without the forced tool call, or an empty final answer cut off by
  `length`. Benign default padding (schema-declared defaults, the natural
  zero value of an optional property, and the shell `cwd` context field) is
  stripped before comparison. Oracle failures, schema failures, and server
  failures are never retried.
- Suite budget: 14 minutes 30 seconds; a hard abort still reports all
  failures collected so far.
- Tool results fed back to the model are capped at 4000 characters per call.

## How to run it

```bash
OPENROUTER_API_KEY=<key in the environment> \
env -u OPENROUTER_MODEL cargo test --test openrouter_e2e \
  -- --ignored --nocapture --test-threads=1
```

- The suite is **ignored by default** — ordinary `cargo test` runs stay
  offline, secret-free, and cost-free.
- Invocation without `OPENROUTER_API_KEY` fails clearly; the gating cannot
  silently skip.
- The key is read from the environment at runtime and is never logged,
  printed, or written anywhere. The summary prints only case metadata,
  request IDs, resolved model names, token usage, and elapsed times.

## Most recent verified acceptance run

The following numbers characterize the latest historical run of the suite
(2026-08-14). They are a record of that run, not a guarantee for future
runs: model names, token usage, and timing vary with OpenRouter's routing
and the resolved model version. The **repeatable guarantees** are the
harness invariants above (coverage assertions, argument guard, oracles,
bounds, retry policy).

| Metric                               | Value                                             |
| ------------------------------------ | ------------------------------------------------- |
| Requested model alias                | `~deepseek/deepseek-v4-flash-latest`              |
| Actual resolved model                | `deepseek/deepseek-v4-flash-0731`                 |
| Cases executed                       | 66 (29 filesystem, 16 memory, 11 fetch, 10 shell) |
| MCP tool calls                       | 66                                                |
| Full roundtrips                      | 66                                                |
| Real OpenRouter HTTP attempts        | 135 (including retries)                           |
| Transport retries                    | 1                                                 |
| Prompt/resource consumption requests | 2 (fetch prompt, memory resource)                 |
| Tokens (in / out)                    | 54,878 / 11,647                                   |
| Total elapsed                        | 404 s                                             |

All 66 cases passed with the programmatic oracles; the fetch prompt and the
memory resource were consumed successfully in real requests.

## Metrics captured

Per case: case ID, server, tool, status (ok/deviation/error), request IDs,
actual (resolved) model, token usage, elapsed, MCP status, retries.
Aggregate: real HTTP attempt count including retries, MCP tool calls, full
roundtrips, prompt/resource consumption requests, tokens in/out, retries.
Never printed: API key, authorization headers, raw response bodies.

The full per-tool case matrix is maintained in the [Coverage matrix](/coverage).