1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
//! Declaration-key and parse-error-message redaction: the credential-shape gate applied to a
//! value that is not always a URL (an opaque label, a manifest declaration key, or a parser's
//! error message), unlike [`super::url`]'s URL-specific redaction.
use ;
/// Redacts a [`crate::ecosystem::BlockedRegistryOccurrence::declaration_key`] value for safe
/// inclusion in a client-visible diagnostic message.
///
/// `key` is not always a URL: most ecosystems set it to a short opaque label (`"top-level"`,
/// `"source:Blocked"`, `"scope:@myorg"`), and only some set it to a real registry URL, sometimes
/// prefixed with an opaque label of the ecosystem's own choosing
/// (`"source:https://user:pass@host/index"`). Running [`super::url::url_for_tracing`]'s aggressive
/// text-scan fallbacks unconditionally would mangle a label like `"scope:@myorg"` into
/// `"***@myorg"` (#981), so redaction only runs when `key` shows real URL structure
/// (`is_authority_bearing_url`) or a credential shape of its own: at least one `@` whose own
/// preceding segment (back to the previous `@`, or `key`'s own start) has a credential-shaped
/// `:` once its own trailing `:` is stripped (`segment_has_credential_colon`, checked
/// per-segment so a decoy trailing `label:@...` segment can't shadow an earlier real credential
/// — #993 M5). Stripping the trailing `:` first is what keeps `"scope:@myorg"` — an
/// empty-username userinfo shape used as an opaque label, not a credential — from matching,
/// since after stripping it the segment is just `"scope"`, with no `:` left to trip the check.
/// This is a shape check, not a separator check: it has no opinion on `/` at all, so it catches
/// a credential regardless of what separates the opaque label from it
/// (`"source:user:ghp_SECRET@host"`, `"source:feed/user:hunter2@host"`,
/// `"source:feed//user:hunter2@host"`, even a percent-encoded
/// `"source:feed%2F%2Fuser:hunter2@h"`, since the scan only looks at `:`/`@`, never `/`), while
/// never mangling a `/`-containing label that has no `@` at all (`"source:feed//mirror"`,
/// `"component-host:gitlab.example.com//group"` are left untouched — #993 S2, a regression in
/// an earlier, separator-based version of this gate). Every other key takes the plain
/// query/fragment-strip fallback instead.
///
/// Two accepted trade-offs, both favoring over-redaction over under-redaction per this module's
/// own stated design (see [`super::url::redact_userinfo`]'s doc):
/// - Once triggered, the actual redaction ([`super::url::url_for_tracing`]) can still destroy more of `key`
/// than just the credential — e.g. `"source:feed//user:hunter2@h"` redacts to `"***@h"`,
/// losing the `"source:feed//"` label entirely — since the fallback scan this reaches has no
/// way to tell where the opaque label ends and true userinfo begins once neither
/// `is_authority_bearing_url` nor a real authority is available to anchor on. This defeats
/// `declaration_key`'s own disambiguation purpose (see its doc, and
/// `build_blocked_registry_diagnostic`'s) for that one value.
/// - A non-credential `label:text@text`-shaped key is redacted too, since the gate cannot tell
/// it apart from a real credential — e.g. `"source:contoso@internal"` redacts to
/// `"***@internal"` even though `contoso` is just a source name, not a password. In practice
/// only a free-text label an ecosystem builds from unvalidated user input (e.g. NuGet's
/// `format!("source:{}", entry.key)` from a `NuGet.config` `<add key>` attribute) is likely to
/// contain an `@` at all — most ecosystems' own fixed/opaque labels cannot.
///
/// This is a best-effort heuristic gate, not a formally verified one: it closes every
/// known-realistic and known-adversarial leak shape found so far (#981, #993), but a
/// sufficiently unusual `key` could in principle still slip past both `is_authority_bearing_url`
/// and this shape check.
///
/// # Examples
///
/// ```
/// use deps_core::redact::redact_declaration_key;
///
/// assert_eq!(
/// redact_declaration_key("https://user:hunter2@10.0.0.1/index"),
/// "https://***@10.0.0.1/index"
/// );
/// assert_eq!(
/// redact_declaration_key("source:https://user:hunter2@10.0.0.1/v3/index.json"),
/// "source:https://***@10.0.0.1/v3/index.json"
/// );
/// assert_eq!(redact_declaration_key("source:Blocked"), "source:Blocked");
/// assert_eq!(redact_declaration_key("scope:@myorg"), "scope:@myorg");
/// // A single `/` (not `//`) still redacts — this is a credential-shape gate, not a
/// // separator-substring gate.
/// assert_eq!(
/// redact_declaration_key("source:feed/user:hunter2@nuget.internal"),
/// "***@nuget.internal"
/// );
/// // No `@` at all: never redacted, even with a `/`-heavy label.
/// assert_eq!(
/// redact_declaration_key("source:feed//mirror"),
/// "source:feed//mirror"
/// );
/// // A control character survives the credential-shape gate (no `@` here at all) but is still
/// // swept, since it could otherwise forge a log line on its own (#1469).
/// assert_eq!(
/// redact_declaration_key("lodash\nERROR deps_lsp: FORGED"),
/// "lodash ERROR deps_lsp: FORGED"
/// );
/// ```
/// Replaces every Unicode `Cc` (Control), `Cf` (Format), `Zl` (Line Separator), and `Zp`
/// (Paragraph Separator) character in `s` with a single space, borrowing when `s` has none
/// (#1242, #1246).
///
/// `Cc` covers ASCII/C1 control characters, including `\n`/`\r` and raw ANSI escape
/// (`\x1B`) bytes, that could otherwise splice new lines into a single-line rendering
/// (table row, JSON string, SARIF message) or forge terminal escape sequences. `Cf`
/// additionally covers invisible formatting characters with no glyph of their own —
/// bidirectional overrides (e.g. U+202E RIGHT-TO-LEFT OVERRIDE, the Trojan Source vector),
/// zero-width joiners/spaces (U+200B, U+200C, U+200D), and the byte-order mark (U+FEFF) —
/// which can visually reorder or hide text without tripping a `Cc`-only check. `Zl`/`Zp`
/// (U+2028 LINE SEPARATOR, U+2029 PARAGRAPH SEPARATOR) are neither `Cc` nor `Cf`, but are
/// still line terminators for JS/`eval` consumers of `--format json` output and are treated
/// as breaks by some editor renderers (critic follow-up M1, #1246).
///
/// This is deliberately wider than [`crate::lsp_helpers::escape_markdown`] and
/// [`crate::lsp_helpers::markdown_code_span`]'s own narrow, explicit bidi/invisible-character
/// list (`is_markdown_unsafe`, #1248): those helpers also run on registry-supplied hover
/// *descriptions*, where legitimate right-to-left text carries real `Cf` marks (e.g. U+200F
/// RIGHT-TO-LEFT MARK, U+061C ARABIC LETTER MARK) and emoji ZWJ sequences carry U+200D, so
/// treating the whole `Cf` category as unsafe there would mangle genuine Arabic/Hebrew text
/// or emoji. A manifest-declared package/coordinate *name*, by contrast, has no legitimate
/// use for any `Cf`/`Zl`/`Zp` character, so this dedicated helper — for name-shaped values
/// only — can safely treat the whole categories as unsafe.
///
/// # Examples
///
/// ```
/// use deps_core::redact::sanitize_invisible;
///
/// assert_eq!(sanitize_invisible("left-pad"), "left-pad");
/// assert_eq!(sanitize_invisible("a\nb\rc"), "a b c");
/// assert_eq!(sanitize_invisible("bidi\u{202E}gnp.exe"), "bidi gnp.exe");
/// assert_eq!(sanitize_invisible("a\u{2028}b\u{2029}c"), "a b c");
/// ```
/// Whether `c` belongs to the Unicode `Cc`/`Cf`/`Zl`/`Zp` general categories
/// [`sanitize_invisible`] sweeps.
///
/// `pub(crate)` rather than private so `lsp_helpers::mod`'s drift-guard test (#1323) can
/// scan the full Unicode code space and assert that every character this predicate
/// flags is either blocked by `is_markdown_unsafe` or in that function's documented,
/// named exempt set — without duplicating this category logic in the test itself,
/// which would defeat the point of a drift guard.
pub
/// Longest prefix (in bytes, not chars — see [`redact_parse_error_for_log`]'s slicing) of a
/// redacted parse-error message logged, post-redaction, by [`redact_parse_error_for_log`].
///
/// `pub`, not `pub(crate)`: `deps_pypi::parser::truncate_for_log` (a thin wrapper delegating to
/// [`redact_parse_error_for_log`]) mirrors this value for its own test fixtures rather than
/// hardcoding a second `200` that could silently drift from this one.
pub const MAX_PARSE_ERROR_LOG_BYTES: usize = 200;
/// Redacts, then bounds, a parse-error's `Display` text before it reaches a log sink.
///
/// `toml_span::Error` and `yaml_rust2::ScanError` can both embed a manifest/lockfile's
/// offending duplicate key or table name verbatim in their `Display` output, and that name can
/// itself be a credential-bearing URL (#1240). [`redact_declaration_key`] runs first, then the
/// result is truncated to a bounded prefix — in that order, since truncating first could cut the
/// string exactly at the boundary the credential-shape scan depends on, leaking a credential
/// that straddles the cut. Truncation is an independent concern from redaction: the file `raw`
/// derives from can be as large as the ~10 MB read cap
/// ([`crate::fs_probe::read_to_string_capped`]'s bound), so an unbounded
/// `tracing::warn!`/`debug!`/`error!` call would be a synchronous multi-megabyte write to a (by
/// default, unbuffered, stderr-backed) log sink — this mostly matters on the benign path, since
/// once redaction actually fires it collapses `raw` to a short fixed-shape string well under the
/// cap. Falls back to `raw` unchanged (as a `Cow::Borrowed`, so a caller that just needs to read
/// or forward the result avoids a second clone) when redaction was a no-op and it's already
/// short enough.
///
/// [`redact_declaration_key`] itself returns a `Cow`, so this function's own no-op path never
/// re-allocates on top of it (#1317) — it just forwards `redact_declaration_key`'s own
/// `Cow::Borrowed`. This is guaranteed allocation-free only for `redact_declaration_key`'s plain
/// opaque-key fallback (no `?`/`#`, no URL/credential shape); its authority-bearing,
/// credential-free case (e.g. a plain registry URL with no query) still allocates internally
/// before that `Cow::Borrowed` short-circuit discards the allocation (code-review follow-up,
/// #1317) — the win there is one fewer allocation, not zero.
///
/// This is the single shared implementation behind both `deps-core`'s own parse-error sinks and
/// `deps_pypi::parser::truncate_for_log`'s delegation to it — see CLAUDE.md's cross-ecosystem
/// rule: the same redact-then-truncate shape was independently needed in ≥2 crates (#1228,
/// #1239, #1240), so it lives here once rather than being reimplemented per-crate.
///
/// # Examples
///
/// ```
/// use deps_core::redact::redact_parse_error_for_log;
///
/// assert_eq!(
/// redact_parse_error_for_log(
/// "duplicate key: `https://svcacct:ghp_SUPERSECRETTOKEN123@pkg.internal.corp/x`"
/// ),
/// "duplicate key: `https://***@pkg.internal.corp/x`"
/// );
/// // Benign errors are left unchanged — the credential-shape gate does not fire on them.
/// assert_eq!(redact_parse_error_for_log("duplicate key: `serde`"), "duplicate key: `serde`");
/// ```
/// Redacts `e` via [`redact_parse_error_for_log`] and wraps the result as the boxed error
/// [`crate::error::DepsError::ParseError`]'s `source` field expects.
///
/// Centralizes the whole "redact, then wrap into an `io::Error`" step — not just the redaction
/// itself — as one call, `deps_core::redact::parse_error_source(&e)`, in place of
/// `Box::new(std::io::Error::other(redact_parse_error_for_log(&e.to_string())))` repeated at
/// each `toml_span`/`yaml-rust2` `map_err` site. A future ecosystem crate adding a new parse-error
/// site is forced through the safe path by construction, rather than being able to copy an old,
/// unfixed `Box::new(std::io::Error::other(e.to_string()))` from elsewhere and reintroduce #1240.
///
/// # Examples
///
/// ```
/// use deps_core::redact::parse_error_source;
///
/// let source = parse_error_source(&"duplicate key: `serde`");
/// assert_eq!(source.to_string(), "duplicate key: `serde`");
/// ```
/// [`redact_parse_error_for_log`]'s owned-input counterpart: `raw` is already an owned
/// allocation the caller has no further use for, so the no-op path returns it directly instead
/// of cloning a borrow of it via `Cow::into_owned` (#1317) — the exact allocation
/// [`parse_error_source`] used to pay twice for on the benign (no-credential) path.
/// The segment-bounded `@`/`:` scan shared by [`redact_declaration_key`] (decides whether a
/// non-URL label needs redacting at all) and [`is_credential_or_query_bearing`] (decides whether an
/// outbound value must be rejected outright) — extracted so the two callers can't drift apart
/// on what counts as "looks like a credential" (#1206).
///
/// See [`redact_declaration_key`]'s own doc for the exact segment/trim rule this implements.
/// Whether `value` carries credential-shaped userinfo of its own, or an untrusted
/// query/fragment component that could hide one.
///
/// Detects, in order:
/// - a `?`/`#` anywhere in `value` (#1206 S2) — a query string is exactly where a token can
/// travel (`?token=...`, `?access_token=...`) without ever taking the `user:pass@host`
/// shape the checks below look for, and [`super::url::url_for_tracing`] already treats *any* query or
/// fragment as untrusted-by-default (it drops both unconditionally, per #866/#858) — this
/// function applies that same always-suspect rule to the reject-before-search decision, not
/// just to what gets logged afterward;
/// - for a real URL authority (`host()` is `Some`), whether [`super::url::redact_userinfo`] would change
/// `value` at all — not just a non-empty username/password on the authority itself. A
/// credential-shaped `user:pass@host` span can also sit *after* the authority, later in the
/// path (e.g. `https://mirror.example/redirect/user:pass@evil.com`, code-review round 1
/// #1206 finding 1): checking only `url.username()`/`url.password()` missed this, since that
/// pair only reflects the authority's own userinfo. Delegating to `redact_userinfo` reuses
/// its existing authority-then-tail scan instead of re-deriving it, and is deliberately
/// scoped to the `host().is_some()` case alone — see the over-rejection note below for why
/// this isn't applied unconditionally to every input;
/// - for text with no parseable URL structure (e.g. a `.package(url: "...")` literal with its
/// scheme already stripped before reaching a registry-search query), the same
/// segment-bounded credential-colon shape [`redact_declaration_key`]'s fallback scan uses
/// (`has_credential_shape`).
///
/// Unlike `is_authority_bearing_url`, this is a genuine credential-detection predicate, not
/// "does this parse as a URL at all" — an ordinary credential-free, query-free URL
/// (`https://host/path`) returns `false` here.
///
/// Narrower than [`super::url::url_for_tracing`]'s own redaction coverage in one specific way: a
/// scheme-stripped, colon-less, token-only userinfo (e.g. `ghp_TOKEN@host/path`, with no `:`
/// anywhere) is not flagged here, even though `url_for_tracing` would still redact it via
/// `find_token_prefix_at`'s prefix sniffing. Not widened to match: probe-testing found that
/// treating "the redactor would change this string" as the reject criterion false-positives on
/// ordinary non-credential input containing a bare `@` with no colon at all — a Maven
/// coordinate like `com.google.guava:guava` (no `@`, unaffected) is fine, but an SSH-style Git
/// remote like `git@github.com:apple/swift-nio.git` would trip a token-prefix-agnostic
/// widening. The `user:pass@`/`label:value@` colon-shaped case this function does catch covers
/// every shape in the actual #1206 repro family (a full `.package(url:)` literal, with or
/// without its scheme).
///
/// Conversely, delegating to `redact_userinfo` for the `host().is_some()` case inherits that
/// function's own documented over-redaction: a colon in the path with no `@` at all (e.g.
/// `https://10.0.0.1/v1/items:search`, a REST-style path segment, no credential involved) is
/// still flagged here, because `redact_userinfo` still rewrites it. Accepted per this module's
/// stated over-redact/over-reject-over-leak trade-off (the same one the `?`/`#` check above
/// already applies) — an occasional false-positive-rejected completion keystroke costs far less
/// than a missed credential leak, and this narrower delegation (scoped to values that already
/// parse as a real URL authority) is far more targeted than falling through to
/// `has_credential_shape` would be, which over-triggers on *any* `scheme://...@...` value purely
/// from the scheme's own leading colon (verified while fixing this finding: naively falling
/// through to `has_credential_shape` for every `host().is_some()` value flagged an ordinary
/// scoped-package-style path like `https://github.com/@babel/core` with no credential anywhere).
///
/// Intended as a reject-before-search gate (issue #1206): a value this function flags must
/// never reach an outbound registry-search query or an unredacted `tracing` field — a redacted
/// value is not a useful search term, so the caller should drop the request entirely rather
/// than redact-and-search.
///
/// # Examples
///
/// ```
/// use deps_core::redact::is_credential_or_query_bearing;
///
/// assert!(is_credential_or_query_bearing(
/// "https://user:hunter2@registry.example/simple"
/// ));
/// // Scheme already stripped by the caller (e.g. Swift's `.package(url:)` completion) — no
/// // longer parses as a URL, but still shows the same credential shape.
/// assert!(is_credential_or_query_bearing(
/// "deploy:AUDITSENTINEL0000@git.internal.corp/team/x.git"
/// ));
/// // A query string can carry a token with no userinfo shape at all.
/// assert!(is_credential_or_query_bearing(
/// "https://github.com/apple/swift-nio?token=SECRET"
/// ));
/// // A credential-shaped span after the authority, later in the path, is caught too.
/// assert!(is_credential_or_query_bearing(
/// "https://mirror.example/redirect/user:pass@evil.com"
/// ));
/// assert!(!is_credential_or_query_bearing("apple/swift-nio"));
/// assert!(!is_credential_or_query_bearing("https://registry.example/simple"));
/// ```