Expand description
Shared proxy / HTTP utilities.
ProxyMode, ProxyConnector, and the low-level get / post helpers
are defined here once and re-exported to every scraper module.
This eliminates the four near-identical copies that existed before.
Structs§
- Proxy
Connector - A hyper
Connectorthat always dials a fixed upstream proxy host:port, regardless of the target URI. The real target URI is carried in theHostheader (HTTP/1.1 CONNECT-less plain-HTTP proxy protocol).
Enums§
- Proxy
Mode - How to route outbound HTTP requests.
Constants§
- USER_
AGENT - Use a realistic browser UA everywhere so the target server does not
reject requests. (Previously
room_list.rsused a different string, causing inconsistent behaviour.)
Functions§
- extract_
jsessionid - Extract JSESSIONID from a header map’s
Set-Cookievalues. - resolve_
url - Resolve
location(possibly relative) againstbase. - send_
follow_ redirects - Follow up to
MAX_REDIRECTSHTTP 3xx responses automatically. - send_
once - Send a single GET request and return
(status, headers, body). - url_
encode - Percent-encode a string for use as a query parameter value.
Uses
percent-encodingcrate withNON_ALPHANUMERICset (safe for all values).