Skip to main content

Module vendor_quirks

Module vendor_quirks 

Source
Expand description

Vendor-specific proxy URL quirks.

§Why vendor-specific quirks matter

Most public proxy providers serve plain HTTP on a port that looks like an HTTPS port. Using https:// against such a proxy causes the TLS layer to attempt a handshake on top of an already-running TLS session, producing BoringSSL’s WRONG_VERSION_NUMBER error. The 2026 guide flags this as a high-frequency footgun (docs/dev/project/scraping-guide-2026-llm-context.md L2840, the “Crawlera/Zyte proxy bug”): guide flags this as a high-frequency footgun (docs/dev/project/scraping-guide-2026-llm-context.md L2840, the “Crawlera/Zyte proxy bug”):

Port 8011 speaks plain HTTP. Both http:// and https:// keys must use http:// scheme. Using https:// causes BoringSSL WRONG_VERSION_NUMBER (TLS-over-TLS failure). Fix: 'https': 'http://key:@proxy.crawlera.com:8011/'

Operators hit this trap on three common providers:

  • Crawlera / Zyte Smart Proxy Manager (*.crawlera.com:8011, *.zyte.com:8011) — must be http:// even when scraping https:// targets.
  • Bright Data brd.superproxy.io:22225 — username must follow the brd-customer-<id>-session-<session_id> pattern; missing the -session-<id> suffix silently collapses all traffic into the default session pool.
  • IPRoyal residential gateway — username must carry a country flag (e.g. user-country-US) for the egress IP to honour the request.

VENDOR_QUIRKS encodes the four documented cases as a const slice so the table is zero-cost at runtime; check walks the slice using pure pointer/length compares and returns all matches in a small Vec. Hard-error quirks (like Crawlera 8011 + https://) are surfaced at ingest time by validate_proxy_url, which rejects the URL outright; warning-severity quirks are logged and the URL is accepted.

§Security note

Quirks match on host:port only — the password component of the URL is never inspected, logged, or echoed in any error or warning. The quirk descriptions are static &'static str slices with no credentials baked in.

§Hot path

check is a pub fn (the return type is Vec<QuirkMatch> per the task spec). In the common case where the URL host is not in the built-in table, the function returns an empty Vec::new() without iterating the table — the Vec allocation is skipped entirely. The table itself is a const slice, so there is no I/O, no locks, and no parsing beyond the ProxyUrl construction done by the caller.

§Example

use stygian_proxy::vendor_quirks::{check, ProxyUrl, QuirkSeverity};

// The classic Crawlera 8011 + https:// footgun.
let url = ProxyUrl::parse("https://user:pass@proxy.crawlera.com:8011")
    .expect("parses");
let matches = check(&url);
assert_eq!(matches.len(), 1);
assert_eq!(matches[0].severity, QuirkSeverity::Error);
assert_eq!(matches[0].required_scheme, stygian_proxy::vendor_quirks::Scheme::Http);

Structs§

ProxyUrl
A parsed proxy URL — the canonical input shape for check.
QuirkMatch
A single vendor-quirk match produced by check.
VendorQuirk
A vendor-specific proxy URL rule, declared as a const table record.

Enums§

ParseError
Errors emitted by ProxyUrl::parse.
QuirkSeverity
Severity classification for a QuirkMatch.
Scheme
URL scheme of a proxy endpoint.

Constants§

BRD_SUPERPROXY_QUIRK
Bright Data residential / datacenter super-proxy on port 22225.
CRAWLERA_8011_QUIRK
The first hard-error quirk — Crawlera port 8011 is plain HTTP.
IPROYAL_QUIRK
IPRoyal residential traffic requires a country flag in the username.
VENDOR_QUIRKS
Every built-in VendorQuirk, in declaration order.
ZYTE_8011_QUIRK
Zyte Smart Proxy Manager port 8011 has the same plain-HTTP trap as CrawleraZyte operates the same upstream port range.

Functions§

check
Returns every VendorQuirk that matches url.