Expand description
Vendor-specific proxy URL quirks.
§Why vendor-specific quirks matter
Most public proxy providers serve plain HTTP on a port that looks
like an HTTPS port. Using https:// against such a proxy causes the
TLS layer to attempt a handshake on top of an already-running TLS
session, producing BoringSSL’s WRONG_VERSION_NUMBER error.
The 2026 guide flags this as a high-frequency footgun
(docs/dev/project/scraping-guide-2026-llm-context.md L2840, the
“Crawlera/Zyte proxy bug”):
guide flags this as a high-frequency footgun
(docs/dev/project/scraping-guide-2026-llm-context.md L2840, the
“Crawlera/Zyte proxy bug”):
Port 8011 speaks plain HTTP. Both http:// and https:// keys must use http:// scheme. Using https:// causes BoringSSL WRONG_VERSION_NUMBER (TLS-over-TLS failure). Fix:
'https': 'http://key:@proxy.crawlera.com:8011/'
Operators hit this trap on three common providers:
Crawlera/ZyteSmart Proxy Manager (*.crawlera.com:8011,*.zyte.com:8011) — must behttp://even when scrapinghttps://targets.Bright Databrd.superproxy.io:22225— username must follow thebrd-customer-<id>-session-<session_id>pattern; missing the-session-<id>suffix silently collapses all traffic into the default session pool.IPRoyalresidential gateway — username must carry a country flag (e.g.user-country-US) for the egress IP to honour the request.
VENDOR_QUIRKS encodes the four documented cases as a const slice
so the table is zero-cost at runtime; check walks the slice using
pure pointer/length compares and returns all matches in a small
Vec. Hard-error quirks (like Crawlera 8011 + https://) are
surfaced at ingest time by validate_proxy_url,
which rejects the URL outright; warning-severity quirks are logged
and the URL is accepted.
§Security note
Quirks match on host:port only — the password component of the URL
is never inspected, logged, or echoed in any error or warning. The
quirk descriptions are static &'static str slices with no
credentials baked in.
§Hot path
check is a pub fn (the return type is Vec<QuirkMatch> per the
task spec). In the common case where the URL host is not in the
built-in table, the function returns an empty Vec::new() without
iterating the table — the Vec allocation is skipped entirely. The
table itself is a const slice, so there is no I/O, no locks, and
no parsing beyond the ProxyUrl construction done by the caller.
§Example
use stygian_proxy::vendor_quirks::{check, ProxyUrl, QuirkSeverity};
// The classic Crawlera 8011 + https:// footgun.
let url = ProxyUrl::parse("https://user:pass@proxy.crawlera.com:8011")
.expect("parses");
let matches = check(&url);
assert_eq!(matches.len(), 1);
assert_eq!(matches[0].severity, QuirkSeverity::Error);
assert_eq!(matches[0].required_scheme, stygian_proxy::vendor_quirks::Scheme::Http);Structs§
- Proxy
Url - A parsed proxy URL — the canonical input shape for
check. - Quirk
Match - A single vendor-quirk match produced by
check. - Vendor
Quirk - A vendor-specific proxy URL rule, declared as a
consttable record.
Enums§
- Parse
Error - Errors emitted by
ProxyUrl::parse. - Quirk
Severity - Severity classification for a
QuirkMatch. - Scheme
- URL scheme of a proxy endpoint.
Constants§
- BRD_
SUPERPROXY_ QUIRK Bright Dataresidential / datacenter super-proxy on port 22225.- CRAWLERA_
8011_ QUIRK - The first hard-error quirk —
Crawleraport 8011 is plain HTTP. - IPROYAL_
QUIRK IPRoyalresidential traffic requires a country flag in the username.- VENDOR_
QUIRKS - Every built-in
VendorQuirk, in declaration order. - ZYTE_
8011_ QUIRK ZyteSmart Proxy Manager port 8011 has the same plain-HTTP trap asCrawlera—Zyteoperates the same upstream port range.
Functions§
- check
- Returns every
VendorQuirkthat matchesurl.