1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
//! Write-time secret detection gate.
//!
//! Scans caller-supplied content strings before any storage write. A match
//! causes a hard `RuntimeError::SecretDetected` that names the detector and
//! carries a masked excerpt internally. Its display names the rule and trigger
//! without echoing any candidate text.
//!
//! Scope: **credentials only** — API keys, tokens, private keys, passwords,
//! and connection strings with embedded credentials. General PII (emails,
//! phone numbers, company names) is intentionally NOT blocked.
//!
//! Detection is layered, cheap-first:
//! 1. **Known-prefix / known-shape patterns** — AWS AKIA/ASIA, GitHub tokens,
//! OpenAI `sk-proj-`, Anthropic `sk-ant-`, Stripe live keys, Fly.io tokens,
//! Vercel secrets, Slack `xox*`, JWT triples, PEM private-key headers, Age
//! secret keys, URL userinfo (`scheme://user:pass@`).
//! 2. **High-entropy token heuristic** — base64/hex/base64url runs ≥ 24 chars
//! near a trigger word (key, secret, password, credential, bearer, auth,
//! apikey, api_key, access_key, private_key). A standalone `token` still
//! triggers opaque entropy detection, but does not by itself label a UUID
//! as a credential; compound identifiers such as `tokenizer_*` and
//! `token_count` remain excluded.
//!
//! Credential-shaped labels and assignments dominate the allowlist below.
//! Public VCS revisions and plausible file paths remain exempt in ordinary
//! technical prose, while path segments are still scanned independently.
//!
//! Full exemption rules (hex/UUID/SRI-hash passes, non-ASCII token
//! delimiting, structured-identifier decomposition, trigger word-boundary
//! matching, the underscore-boundary asymmetry between bare trigger words and
//! the word `token`, and the adversarial-corpus rationale for why some
//! false positives are accepted) are documented in full in
//! `docs/api/secret_gate.md#module-level-detection-algorithm` — read that before
//! changing any detection or exemption logic in this file.
//!
//! The caller-visible block message (`SecretMatch`'s `Display` impl) also
//! carries actionable guidance (`block_guidance`) to split or reword the
//! flagged token.
//!
//! A production-corpus replay harness (`corpus_replay`, `#[ignore]`d, run via
//! `KHIVE_REPLAY_DB=<path> cargo test ... -- --ignored --nocapture`) measures
//! the detector's block rate against real note/entity content; see the
//! harness's own output for current numbers rather than a point-in-time count
//! here, which would drift as the corpus changes. A checked-in, sanitized
//! snapshot of that replay (per-detector block counts and sha256 digests of
//! blocked content, never the content itself) lives at
//! `tests/data/secret_gate_corpus_manifest.md`, generated by
//! `corpus_replay::generate_corpus_manifest`.
//! A replay that is identical before and after a change certifies preservation
//! on the corpus population only, never on the shape class the change opens;
//! that class needs its own before/after arms.
use crate;
use scan;
pub use ;
pub
/// Hard-block content from being written.
///
/// Returns `Err(RuntimeError::SecretDetected)` on the first match found, or
/// `Ok(())` if no secret pattern fires.
/// Recursively scan a JSON value for credential-shaped strings.
///
/// Walks every string leaf (object values, array elements, nested objects).
/// Returns `Err(RuntimeError::SecretDetected)` on the first match found.
/// `None` / null / numeric / boolean JSON values are skipped.
/// Scan a string-tagged slice (entity/note tags).
///
/// Each tag string is scanned individually.
/// Name the scope and field a refusal came from.
///
/// A batch verb scans each record and returns on the first refusal, so the
/// caller gets ONE error for N records. Without this the error names only the
/// matched text, which by construction is text the caller cannot find: it sits
/// in whichever sibling record refused, and every other record in the call is
/// rejected with it (khive #2605). Pass through anything that is not a gate
/// refusal unchanged — this adds identity, it does not reclassify.
///
/// `record` names the record the field belongs to, as the caller submitted it:
/// `entity`, `note`, `task`, `proposal`, `message`, indexed when it came from a
/// batch (`note[2]`). It answers where in the submitted payload the writer
/// should look, which is the question a refused writer actually asks.
/// `check` that names where it looked. `record` is the record noun the caller
/// submitted, indexed inside a batch; `field` is the field of it that was scanned.
/// `check_json` that names where it looked. The location is the field holding
/// the JSON, not the path of the string leaf that matched inside it.
/// Scan the supplied note fields in content, name, then properties order.
///
/// Returns the first refusal unchanged, located under `record` (for example,
/// `note[2].properties`). Optional fields are scanned only when present; JSON
/// scanning uses the same structured-value rules as [`check_json_at`].
///
/// This is field scanning only. Callers retain responsibility for property
/// derivation, reserved-key checks, embedding inputs, and write admission, in
/// their existing order.
/// `check_tags` that names where it looked.
// ─── Reserved property keys ──────────────────────────────────────────────────
/// Top-level JSON property key reserved for runtime-owned exemption state.
///
/// No caller may create, replace, merge, or remove this key through any
/// properties-bearing write path — ADR-115 Amendment 1 §3. Reservation binds
/// unconditionally: the runtime does not yet stamp any record with this key
/// (the finalizer that would do so is a separate, later increment), so no
/// caller-supplied occurrence of it can ever be a legitimate echo of
/// persisted state. Only the exact top-level key is reserved; the same
/// spelling nested inside an object *value* is ordinary content and remains
/// subject to [`check_json`], never a posture mutation.
pub const RESERVED_SECRET_GATE_KEY: &str = "khive:secret_gate";
pub const RESERVED_WEB_RECEIPT_KEY: &str = "khive:web_receipt";
pub const WEB_RECEIPT_PROVENANCE_VALUE: &str = "v1";
/// Reject a caller-supplied top-level `khive:secret_gate` property key.
///
/// Call this before any diff, merge, or storage preparation touches
/// caller-supplied `properties` on any properties-bearing write path —
/// create, patch update, or full replace. Returns `Ok(())` when `properties`
/// is absent, is not a JSON object, or does not name the reserved key at the
/// top level.
///
/// This is the shared validator for both the ADR-115 reservation and web
/// receipt provenance. Every properties-bearing generic write path calls it
/// before a merge or full-row replacement.
/// The top-level property keys no caller may set, in the order their refusals are reported.
pub const RESERVED_PROPERTY_KEYS: =
;
/// ADR-115 §4 for a whole-map property write against a persisted row.
///
/// An incoming reserved key whose value is byte-identical to the persisted value is an echo
/// of runtime state and is removed from `incoming`. A reserved key the row does not carry, or
/// one whose value differs (`null` included), is refused. Omitting a reserved key from a
/// whole map is not a removal: whether the persisted value carries forward is the caller's
/// decision, conditioned on the exempted bytes (see [`reserved_properties`]).
pub
/// The reserved entries a persisted property map carries, in [`RESERVED_PROPERTY_KEYS`] order.
pub
/// `properties` with the reserved keys removed: the part of a property map the secret gate
/// scans, and so the part an exemption stamp vouches for.
pub
/// The write-site check for ADR-115 §4 on a whole-row replace of a row that may carry a
/// reserved key. A reserved value may reach the row only as a byte-identical carry of the
/// value `persisted` already holds, and only while the bytes it vouches for are unchanged
/// (`vouched_unchanged`). Every such write site calls this immediately before it builds the
/// write, whatever its preparation checked, so a preparation that drifts cannot persist a
/// stamp over bytes it never covered.
pub
// ─── Tests ───────────────────────────────────────────────────────────────────
// ─── Corpus replay harness (manual, opt-in) ─────────────────────────────────
//
// Measures how many real note/entity strings the gate blocks, so a detector
// change can be evaluated against production content rather than intuition
// (see the module doc). Opens the target database
// STRICTLY read-only (`SQLITE_OPEN_READ_ONLY`) and never mutates it. Point
// `KHIVE_REPLAY_DB` at a copy or a live KG database file path; the harness
// never writes, locks aggressively, or deletes anything.
//
// Run with: `KHIVE_REPLAY_DB=/path/to/khive.db cargo test -p khive-runtime \
// --release -- --ignored --nocapture corpus_replay`