# Prompt-cache audit scripts
[Feature docs index](README.md) · [Repository README](../../README.md)
## Purpose
Maintainer scripts under `scripts/` check prompt-cache readiness without changing provider runtime code. Static audit is offline by default. Live checker is explicit opt-in for networked cold/warm experiments.
## Static audit: `scripts/audit_prompt_cache.py`
Use this script before and after prompt-cache provider work:
```sh
python3 scripts/audit_prompt_cache.py --repo .
python3 scripts/audit_prompt_cache.py --repo . --json > /tmp/magi-code-prompt-cache-audit.json
python3 scripts/audit_prompt_cache.py --repo . --strict
python3 scripts/audit_prompt_cache.py --self-test
```
Checks cover:
- Stable prompt prefix ordering in agent conversation assembly.
- Obvious ephemeral system-prompt inputs such as timestamps, random ids, session ids, auth/account ids, or cwd-specific runtime state.
- Provider request cache-scope plumbing.
- Custom Responses `prompt_cache_key` support.
- OpenAI Codex cache-scope evidence.
- Chat Completions streaming usage telemetry request.
- Cached-token and cache-write usage parsing.
- Local `ContextCache` separation from provider prompt cache.
- Custom provider classification via `use_responses_endpoint`.
Status meanings:
| `PASS` | Required static evidence exists. |
| `FAIL` | Required implementation evidence is missing or contradicted. |
| `WARN` | Evidence is incomplete, provider-specific, or optional outside strict mode. |
| `N/A` | Check does not apply to detected provider shape. |
Exit codes:
| `0` | Audit completed with no `FAIL` findings. `WARN` allowed unless `--strict` is set. |
| `1` | Audit completed with `FAIL` findings, or `--strict` promoted `WARN` to failing exit semantics. |
| `2` | Usage error, missing/unreadable source file, invalid fixture/body data, or self-test failure. |
`--json` emits one object with `schema_version`, `repo`, `generated_at`, `overall_status`, `strict`, and `findings`. Each finding includes stable fields such as `id`, `status`, `severity`, `category`, `provider`, `file`, `symbol`, `evidence`, `expected`, `impact`, `suggested_fix`, and `references`.
Current observed static result: `Overall: WARN` with `PASS=11`, `WARN=1`, `FAIL=0` (generated by `python3 scripts/audit_prompt_cache.py --repo .`). `codex.cache_scope` is WARN because public documentation does not verify `prompt_cache_key` acceptance or scope for `https://chatgpt.com/backend-api/codex/responses`; `--strict` therefore exits `1` until authoritative evidence exists.
Implemented static evidence includes:
- `request.cache_scope`: `ProviderRequest` carries a crate-private stable prompt cache key.
- `responses.prompt_cache_key`: custom Responses bodies emit `prompt_cache_key` when request scope exists.
- `codex.cache_scope`: Codex Responses body contains `prompt_cache_key` source evidence, but backend acceptance and scope remain unverified; this is not live cache-hit proof.
- `usage.cached_tokens.chat`: Chat Completions `prompt_tokens_details.cached_tokens` maps to `Usage.cache_read`.
- `chat.stream_usage`: streaming Chat Completions requests include `stream_options.include_usage` so providers can return usage telemetry.
- Agent session runs derive bounded cache scope as `magi-code-session-{first_32_hex_sha256(session.id())}`. Raw session ids, cwd, account data, timestamps, random values, and prompt text are not included.
Chat Completions bodies intentionally do not emit `prompt_cache_key`; only Responses-compatible request shapes receive that field.
- Do not interpret `PASS` findings as proof of live provider cache hits. Static audit proves source evidence only; Codex cache scope is explicitly unresolved.
## Live checker: `scripts/check_prompt_cache_live.py`
Use this script only when you intentionally want a networked provider check:
```sh
python3 scripts/check_prompt_cache_live.py --help
python3 scripts/check_prompt_cache_live.py \
--url https://example.invalid \
--shape responses \
--body-file scripts/fixtures/prompt_cache_audit/live_offline_body.json
python3 scripts/check_prompt_cache_live.py \
--allow-network \
--url "$URL" \
--shape responses \
--body-file /tmp/prompt-cache-live-body.json \
--bearer-env OPENAI_API_KEY \
--repeat 2 \
--expect-warm-cache
```
Safety behavior:
- Network is denied unless `--allow-network` is present.
- Without `--allow-network`, send attempts exit `2` before reading secrets.
- Secrets are read only from named environment variables such as `--bearer-env` and `--account-id-env`.
- Script never reads `~/.magi-code/auth.json` or provider settings automatically.
- Secret-bearing `--header` names such as `authorization`, `api-key`, cookies, and `chatgpt-account-id` are rejected; use env flags instead.
- Output redacts bearer tokens, API keys, cookies, tokens, and account ids.
- Request body is read once after the network gate, kept as bytes, and reused for each repeat. Output reports `body_sha256` and `body_bytes`.
- JSON and SSE `data:` responses are parsed for cached-token fields. Raw request headers are not logged.
Supported shapes:
| `responses` | `usage.input_tokens_details.cached_tokens`, `response.usage.input_tokens_details.cached_tokens`, root and nested cache-write locations |
| `chat-completions` | `usage.prompt_tokens_details.cached_tokens`, `usage.prompt_tokens`, `usage.completion_tokens`, `usage.total_tokens` |
| `codex` | Responses-style SSE/JSON fields plus Codex headers supplied through safe env flags; cache-key backend contract remains unverified. |
Exit codes:
| `0` | Invocation and expectations succeeded. |
| `2` | Usage, offline gate, body, header, or self-test error. |
| `3` | Network/auth/provider response failure, or `--expect-warm-cache` found no positive cached-token field on warm attempt. |
## Verification
Credential-free checks:
```sh
python3 scripts/audit_prompt_cache.py --help
python3 scripts/check_prompt_cache_live.py --help
python3 scripts/audit_prompt_cache.py --self-test
python3 scripts/check_prompt_cache_live.py --self-test
python3 scripts/audit_prompt_cache.py --repo .
python3 scripts/audit_prompt_cache.py --repo . --strict
python3 scripts/audit_prompt_cache.py --repo . --json
```
Observed static audit result is `Overall: WARN` with one unresolved Codex finding; non-strict audit exits `0`, while `--strict` exits `1` by design until authoritative Codex cache-scope evidence exists.
---
[Back to feature docs](README.md) · [Back to repository README](../../README.md)