chromeleon (Rust)
A thin, driver-agnostic Rust client for Chromeleon's per-context proxy
handshake. No browser library is a dependency: it works with
chromiumoxide, with headless_chrome, and with
a raw DevTools WebSocket, in async or blocking code.
See HANDSHAKE.md
for the protocol this implements.
Install
[]
= "0.1" # https://crates.io/crates/chromeleon
# optional: the chromiumoxide adapter (needs rustc >= 1.85, chromiumoxide's own floor)
= { = "0.1", = ["chromiumoxide"] }
MSRV 1.75, and CI builds and tests on exactly that toolchain — but read the
caveat, because it is about your dependency resolution, not our code. Newer
releases in this tree require 1.85 (async-lock 3.4.2 declares it; url 2.5.4+
reaches idna_adapter, which is edition 2024 and will not even parse on an old
cargo), and a plain cargo build takes the newest. On 1.75 you therefore need
one of: the Cargo.lock committed here, [resolver] incompatible-rust-versions = "fallback" (cargo ≥ 1.84, so not the cargo that
ships with 1.75), or your own cargo update --precise pins. On any recent
toolchain none of this applies.
Launch, then attach
Rust drives browsers by spawning them and attaching over CDP, so that is the
shape of the launch helper. It merges the WebRTC policy flag a per-context proxy
needs and strips the controller's PROXY_* variables from the browser's
environment — the two things your driver has no way to know about.
use Launcher;
let = new
.remote_debugging_port
.arg
.fingerprint_os // ⚠️ see below — the default is Windows
.captcha // the built-in solver, on
.spawn_and_wait?; // returns the DevTools WebSocket URL
Two things that bite:
- Never pass proxy credentials at launch. The binary fails closed on an
unresolved exit IP, because that would unbind the persona's geo and disable
WebRTC masking. Per-context proxies are launched with no
--proxy-server. - The persona OS defaults to Windows. It is read from the browser's command
line when a context is created, so a launcher that never sets
--fingerprint-osships Windows personas from a Linux host, silently.Launcher::fingerprint_osis the switch.
The handshake
One call, for any async CDP connection:
use new_proxy_context_raw;
let created = new_proxy_context_raw.await?;
let context_id = created.as_str.unwrap;
…or step by step, when your driver creates contexts its own way:
use ;
let reg = proxy_registration.await?;
check_registration?;
let context = browser.create_context.await?; // same bytes, still under the guard
drop; // released only after the create
The guard is the API's whole point: the registration is single-use — consumed by
the create, and consumed even when that create then fails — so Registration is held
across both commands and serializes anything else that wants the same
(connection, server). Different servers on one connection run concurrently,
which is the throughput case:
let = join!;
Synchronous drivers use the same lock through
new_proxy_context_raw_blocking / proxy_registration_blocking.
chromiumoxide
use new_proxy_context;
let = connect.await?; // from Launcher, above
spawn;
let context = new_proxy_context.await?;
let mut target = new;
target.browser_context_id = Some;
let page = browser.new_page.await?;
Target.setProxyCredentials is not in chromiumoxide's generated protocol and it
has no untyped send; oxide::RawCommand is the escape hatch, and
oxide::send_raw / send_raw_page expose it. oxide::browser_config curates
chromiumoxide's own default flags (it adds --enable-automation and
--lang=en_US, neither of which belongs on this browser) — but note it cannot
strip the controller's PROXY_*, because BrowserConfig only adds
environment variables. That is why the example above launches with Launcher
and attaches. Full example: examples/chromiumoxide_proxy_context.rs.
Captcha solving
Launch with .captcha(true) and the built-in reCAPTCHA/hCaptcha solver handles
challenges by itself — you do not call it, and the release binary embeds the
models. The Chromeleon CDP domain only reports the lifecycle: enable it on a
page session and parse the events.
use ;
page_session.send.await?;
while let Some = events.next.await
solver_eval_params(expression, frame_url_contains) runs JS in the solver's
isolated world, which pierces closed shadow roots; its string result arrives
on the SOLVER_EVAL_RESULT event, not as the command's return value.
Three things the other clients get wrong or leave out, checked against the browser source:
methodisaudio(reCAPTCHA),checkbox(hCaptcha/Turnstile),slider(DataDome) orpress-hold(PerimeterX). There is no image mode — image grids and reCAPTCHA v3 are out of scope for the solver.sitekeycarries the embedder host, not a sitekey, for the hCaptcha-checkbox, DataDome and PerimeterX solvers.- The domain exists only when the browser was launched with
--captcha-solver; otherwiseChromeleon.enableis an unknown method. And its events are broadcast to every enabled session in the process, so asolverEvalresult cannot be correlated to the call that produced it.
Page completion
load fires when the first document's subresources are in, and says nothing
about the text you came for. Chromeleon.waitForSettle does: it settles when
the main frame's rendered text has been unchanged for a quiet window,
sampled in the browser process over Blink's inner-text channel — no JavaScript
in the page, no isolated world, nothing registered on the document, which is the
point of it over a MutationObserver a bot wall can see.
use ;
let mut watch = arm.await?; // BEFORE the navigation
page.goto.await?; // committing is enough
let state = watch.wait.await?;
state.outcome // Settled | Timeout | Challenge | Other(_)
state.via // WaitForSettle | NetworkAlmostIdle | Load | Timeout
state.blocked // a bot wall: the classification callers need
state.elapsed_ms; state.text_length; state.http_status; state.navigations;
watch.close.await;
Two phases, not one call, because the ordering mistake is fatal and silent: the
challenge header is recorded at commit time by a tracker attached when the
handler is constructed, so a session attached to an already-committed document
cannot see it and degrades to status-code-only detection without saying so. A
one-shot helper called after goto would miss everything, so this API cannot
spell one. BlockingSettleWatch is the same thing for synchronous drivers; a
complete raw-WebSocket example is examples/settle_wait.rs.
Launch with it on. The domain is registered only under --page-settle —
Launcher::page_settle(true), or page_settle_tuning(SettleTuning::new()…) for
the quiet window, min-chars, timeout, sample interval and shadow-piercing
switches. Without it the command is an unknown method (-32601), not
"settling is off". Like --captcha-solver it is deliberately not in
LAUNCH_ARGS: it changes what the browser does, so it is opt-in.
Why this order
Measured over 960 navigations, 80 sites, 3 rounds:
| direct p50 | direct never | proxied p50 | proxied never | |
|---|---|---|---|---|
networkAlmostIdle |
3,989ms | 1% | 11,285ms | 36% |
Chromeleon.waitForSettle |
7,306ms | 6% | 12,108ms | 7% |
waitForSettle is the primary because through a proxy — which is what this
client is for — networkAlmostIdle never fires on more than a third of loads,
against 7%. On a direct connection networkAlmostIdle is 1.8x faster for the
same median content, which is why it stays available and is the documented
choice for un-proxied work: WaitOptions::new().prefer_lifecycle(true). On
completeness, direct: waitForSettle reproduced the final text exactly on 79%
of loads, networkAlmostIdle 59%, the load event 40%, domcontentloaded 7%.
What the fallback gets right
When the command is unavailable (-32601) or answers something unreadable, the
watch falls back without raising — a page that never settles is a normal
outcome on proxied traffic, not an exception — to a main-frame,
non-stale networkAlmostIdle, then the load event, then Timeout. via says
which, so a fleet quietly running on the weaker signal is visible in a log line
rather than in a month of thin extractions. Two rules it enforces, both of which
were silent wrong answers before they were rules:
- The
about:blankreplay is discarded.Page.setLifecycleEventsEnabledreplays a complete lifecycle for the document the tab is already on, so arming records every loader seen in its first 350ms as stale. Keep them and every fallback signal reads as ~0ms — an instant, wrong "settled". - Subframes never answer for the page.
bbc.com/newsemits lifecycle from 23 frames; first-across-all reportsnetworkIdleat 699ms when the main frame's real value is 6094ms. OnlyframeId == the main framecounts, which is why a session that cannot name its main frame is refused at arm time.
Blocked is not a slow page
state.blocked() is outcome == Challenge or an
http_status in {401, 403, 407, 429, 503} — 12.5% of proxied navigations in
production measurement. Both halves are load-bearing: a wall shorter than
--page-settle-min-chars never reaches the challenge outcome, it reports
timeout with the 403 still on the committed document, and a caller switching
on the outcome alone retries straight back into it.
Supplying the session
This crate depends on no browser library, so a watch cannot own a CDP session
any more than the handshake can own a connection: implement PageSession (or
BlockingPageSession) over your driver. Two methods, and the second is where
the runtime choice stays yours:
API
Launcher |
the browser process: flags merged, PROXY_* stripped |
proxy_registration(conn, proxy) -> Registration |
hold the slot (also _blocking) |
with_proxy_registration(conn, proxy, body) |
the same, closure-shaped |
new_proxy_context_raw(conn, proxy, send) |
both commands over one send |
new_proxy_context_with(conn, proxy, send, create) |
when the driver owns context creation |
ProxySpec |
a parsed proxy; its server is always normalized |
normalize_server, parse_proxy, check_registration |
the pieces, if you want them |
captcha::* |
solver switches, CDP methods, typed events |
settle::SettleWatch / BlockingSettleWatch |
page completion: arm, then wait (also LifecycleTracker on its own) |
settle::settle_launch_args, Launcher::page_settle |
--page-settle and its tuning switches |
perf::sticky_geo_env, perf::BrowserPool |
launch-latency helpers |
oxide::* (feature) |
chromiumoxide adapter: contexts, raw commands, typed events |
proxy is anything IntoProxySpec accepts: a URL string
(http://user:pass@host:port), a ProxySpec, or a (server, username, password) tuple. URL credentials are percent-decoded; credentials you pass
separately are literal.
What this crate refuses to do
Each of these was a silent failure in a real session, which is why it is an error here and not a best effort:
- An unnormalized server.
ProxySpeccan only be built through a constructor that normalizes, so the string you register and the string the context is created with cannot drift apart. - A guessed host. An empty server, a bad port, or a forbidden character in
the host is an error. A driver's own fallback quietly turns
http://host:notaportinto the hosthttp, and the browser then proxies nowhere you meant. - A guessed credential. A URL credential whose escapes decode to non-UTF-8
bytes is
Error::UndecodableCredential, not a replacement character — authenticating with a secret that is nearly yours fails at the proxy, not here, and looks like a network problem. - Dropping credentials. If you pass a server that embeds
user:pass@and no separate credentials, they are taken from the URL rather than discarded.
Parity with the Python and Node clients
All three implement HANDSHAKE.md, and ../conformance/vectors.json holds 94
proxy inputs with the answer every client must give. This client passes all 91
that its types can express (the other three are dynamic-typing cases — a null
server, a non-string credential — that cannot be constructed here).
Twenty of those cases are ones where the Python and Node clients disagree
with each other. The corpus records who was right and why; this client follows
the ruling, so it deliberately differs from each of the older clients on some
inputs. The observed field of each case is the losing behaviour, kept visible
rather than flattened.
Version-dependent behaviour
Up to v151.4 (the binary published as latest) the browser rejects a second
registration for a (connection, server) while one is pending. From v151.5 it
queues, and accepts a credentialsId correlation token —
CredentialsParams::with_credentials_id, oxide::new_proxy_context_with_id,
opt-in because an older binary does not know the field. This client always takes
the lock either way: it cannot know which binary it is talking to, and the lock
costs nothing against a browser that would have queued.
Test
No browser is needed for any of it.