captchaforge 0.2.7

Automatic CAPTCHA detection and multi-strategy solving for chromiumoxide-driven headless browsers (Cloudflare Turnstile, reCAPTCHA v2/v3, hCaptcha, image grids, audio, sliders).
Documentation
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Architecture sweep + community rule pack (2026-05-11)

#### Added
- **Per-detector module split.** `detect.rs` (1442 LOC) → `src/detect/` with one file per built-in detector (`turnstile.rs`, `recaptcha.rs`, `hcaptcha.rs`, `image_captcha.rs`, `audio_captcha.rs`, `pow_captcha.rs`, `slider_captcha.rs`, `canvas_captcha.rs`, `multi_step.rs`, `shadow_dom.rs`, `challenge_page.rs`). New captcha types ship as one self-contained file.
- **`CaptchaProvider` trait + `ProviderRegistry`.** Bundles a detector with its recommended `SolveMethod` ordering. Built-in providers wired via macro. `ProviderRegistry::with_built_in_rules()` returns built-in providers + bundled community rule pack, priority-sorted.
- **TOML rule layer (`detect::rules`).** Community-extensible captcha detectors as data files. Selector / window-global / script-src triggers compile to deterministic, escaped JS probes. Bundled `rules/community.toml` covers AWS WAF Captcha, DataDome, Akamai Bot Manager, PerimeterX/HUMAN, Arkose/FunCaptcha, Geetest v3+v4, Friendly Captcha, MTCaptcha, WordPress math captchas — 10 vendors with zero Rust per vendor.
- **`SolverTelemetry` trait + `SolveEvent`.** One event per solver attempt with outcome (`Success` / `Failure` / `Error` / `Timeout`), time, confidence, domain, captcha kind, method. Default `NoopTelemetry`. Cache hits also fire as Success with `solver: "TokenCache"` so dashboards distinguish cache-served from solved-from-scratch. Install via `chain.with_telemetry(Arc::new(...))`.
- **`TokenCache` with TTL.** Per-`(domain, captcha_type)` solved-token cache (60s default). Chain checks before solving; cache hit returns immediately and reports through telemetry. `chain.cached_solution(info)` exposes the standalone short-circuit for callers without a Page.
- **`PatternStore` persistence.** `save_to_path` / `load_from_path` (atomic JSON via temp+rename), `merge` (cross-instance share, higher `sample_count` wins), `load_or_default` (graceful first-boot path).
- **`auto_solve(page)` top-level convenience.** One-call detect + solve with the bundled provider registry + default chain.
- **CLI binary (`captchaforge`).** Subcommands `inspect-rules` / `detect <url>` / `solve <url>` with `--rules <file>` for additive extra TOML rule packs. Lives in `cli/` workspace member.
- **Per-rule HTML fixtures.** Every bundled rule has positive + negative HTML fixtures under `tests/rule_fixtures/<vendor>/`. The `community_rules` integration test asserts each rule fires on positive, rejects negative, and doesn't cross-fire on any other vendor's negative.
- **Chain ↔ cache ↔ telemetry integration tests.** Verify cache hit fires exactly one Success event with `solver: "TokenCache"`, miss fires nothing, domain isolation works, builder methods install passed instances.

#### Changed
- **`DetectedCaptcha` is `#[non_exhaustive]`.** Adding vendors is no longer a breaking change for downstream `match`. New `Custom(String)` variant lets TOML rules surface vendor identifiers without enum edits.

#### Fixed
- **Custom captchas now route through provider hints.** Previously every solver's `supports()` returned false for `DetectedCaptcha::Custom`, so all 10 TOML-detected vendors detected successfully but got no solver attempt — chain silently returned unsolved. Chain now consults the installed `ProviderRegistry` for Custom kinds and filters solvers by the provider's `recommended_solver_methods`. Install via `chain.with_provider_registry(...)` (`auto_solve` does this for you).

### Legendary push: HTTPS bench, no automation flags, math + PoW solvers (2026-05-11, round 8)

#### Added
- **`solver::MathCaptchaSolver`** — parses math-captcha questions
  ("3 + 5 = ?", "what is two plus four", "12 ÷ 4") from page text and
  types the answer. Handles digit numbers, word numbers (zero-twenty),
  and word operators (plus/minus/times/divided by). Free, ~50ms.
  Restricted to math-flavoured Custom names + PowCaptcha + Image/Canvas
  so it doesn't claim slider/click captchas.
- **`solver::PowCaptchaSolver`** — solves ALTCHA / Friendly Captcha /
  MCaptcha / Cap.dev. Computes SHA-256 PoW in-page via SubtleCrypto
  for ALTCHA (active) or polls the widget's own worker output for
  Friendly / MCaptcha / Cap.dev. No third-party API, no LLM.
- **HTTPS bench server (rcgen self-signed TLS).** Bench fixtures now
  load from `https://localhost:<port>` so vendors that refuse
  non-HTTPS origins (modern Cloudflare / hCaptcha / Google) can
  actually issue tokens against test sitekeys. PREREQUISITE for
  honest auto-pass measurement.
- **Bench chromium config strips automation signals.** Removed
  `--enable-automation` (it was never explicit but chromiumoxide
  defaults add it), added `--exclude-switches=enable-automation`,
  `--disable-blink-features=AutomationControlled`, and the
  `--ignore-certificate-errors` family for the self-signed bench
  TLS.

#### Changed
- **Default chain order** — seven solvers now:
  WaitForToken → Math → Pow → Behavioral → VLM → Audio → ThirdParty.
  Cheapest solvers run first.
- **`Config::build_chain()`** mirrors the same order.

### Stealth module + honest bench results (2026-05-11, round 7)

#### Added
- **`captchaforge::stealth` module** + `apply_stealth(page)` helper.
  Injects CDP `Page.addScriptToEvaluateOnNewDocument` overrides
  for `navigator.webdriver`, `navigator.plugins`,
  `navigator.languages`, `Notification.permission`, `window.chrome`,
  `WebGLRenderingContext.getParameter` (vendor + renderer), and
  `HTMLIFrameElement.contentWindow.chrome`. Without these, modern
  WAFs (Cloudflare, Akamai, PerimeterX/HUMAN, DataDome) detect
  headless chromium *trivially* via `navigator.webdriver === true`
  and refuse to issue a passive-pass token regardless of mouse
  cleverness.
- **PRODUCTION.md "Stealth — required for production" section** —
  documents the surface, when to call `apply_stealth`, and the
  things stealth does NOT cover (UA string, TLS fingerprint, IP
  reputation) that operators must handle separately.
- **`auto_solve()` calls `apply_stealth` defensively** before the
  detection step. Production deployments should still call it
  themselves immediately after `Browser::new_page`, before the
  first navigation — that's the only way to catch first-page
  detection probes.
- **Bench `BrowserPool::reset_page` reapplies stealth** so every
  bench iteration starts with fresh stealth overrides — without
  this, the real-WAF suite reports 0% success on auto-pass
  fixtures purely because Cloudflare/hCaptcha/Google detect headless
  chromium and refuse the passive-pass token.
- **Bench `real_waf` suite uses a 12-second `WaitForTokenSolver`
  budget** (vs 3s default) — production passive Cloudflare can take
  5-10s to issue the token.

### WaitForToken passive-pass solver + honest real-WAF fixtures (2026-05-11, round 6)

#### Added
- **`solver::WaitForTokenSolver`** — first solver in the default
  chain. Polls common vendor response fields (`cf-turnstile-response`,
  `g-recaptcha-response`, `h-captcha-response`, plus invisible
  reCAPTCHA's `grecaptcha.getResponse()` and Friendly Captcha /
  MTCaptcha fields) and returns the token when the vendor populates
  it. Handles the most common production case — passive-pass widgets
  (Turnstile passive mode, reCAPTCHA v3, hCaptcha invisible) — without
  ever clicking, screenshotting, or calling a paid API. 3-second
  default max-wait when used as the chain's lead solver; if the field
  stays empty the chain falls through to behavioural / VLM /
  third-party as before. Cheapest possible solve path.
- **`SolveMethod::AutoPass`** + `#[non_exhaustive]` on the enum so
  future additions don't break downstream `match`. Telemetry events
  from the new solver carry this method tag.
- **`REAL_TURNSTILE_BLOCK` fixture** (Cloudflare always-block test
  sitekey `2x00000000000000000000AB`). Bench expects FAILURE — proves
  the chain doesn't fake-pass against a sitekey that issues no token.
- **`REAL_TURNSTILE_INTERACTIVE` fixture** (Cloudflare force-interactive
  test sitekey `3x00000000000000000000FF`). Honest gap measurement for
  the behavioural solver — passive-pass cannot help here, and any
  success% > 0 means real interactive solving worked.
- **`real_waf` suite gained per-fixture `Expectation`** (AutoPass /
  AlwaysBlock / ForceInteractive). The reported `success` column is
  "outcome matched expectation" — solving a block fixture is an
  INTEGRITY FAILURE, NOT a green row.

#### Changed
- **`CaptchaSolverChain::default_chain()` order**:
  `WaitForToken (3s) → Behavioral → VLM → Audio → ThirdParty`. Five
  solvers; existing tests updated. The cheap solver runs first so
  passive-pass widgets short-circuit the chain before any expensive
  layer fires.
- **`Config::build_chain()`** mirrors the same order so
  `auto_solve()` and the CLI inherit the change.

### Production-ready: bench rewire, real-WAF proof, retry, metrics, security warn, PRODUCTION.md (2026-05-11, round 5)

#### Added
- **`real_waf` bench suite (opt-in)** — drives the production
  `Config::default().build_chain()` against the real
  `challenges.cloudflare.com`, `js.hcaptcha.com`, and
  `google.com/recaptcha/api.js` scripts using each vendor's published
  test sitekey. Asserts the chain returns a token that's actually
  shaped like a real one (rejects the local-fixture `XXXX.DUMMY.TOKEN`
  stubs). Skipped by default; pass `--with-network` (or `--suite real-waf`)
  to enable. Three new fixtures: `REAL_TURNSTILE`, `REAL_HCAPTCHA`,
  `REAL_RECAPTCHA_V2`.
- **`solver::CacheStats` + `TokenCache::stats()` / `reset_stats()`** —
  atomic hit/miss/expired/put/invalidate counters and a `hit_rate()`
  helper. Production deployments scrape `stats()` each minute to plot
  cache effectiveness; a high `expired_misses : hits` ratio is the
  signal that TTL is shorter than the typical re-visit window.
- **Transient-error retry + exponential backoff in `ThirdPartyCaptchaSolver`** —
  3 attempts with 250ms→500ms→1s→2s backoff (capped 4s) on `submit_task`
  and per-poll HTTP. Distinct from the terminal-API-error fail-fast
  already shipped: terminal codes (bad key, zero balance, banned IP)
  short-circuit on first hit; transient HTTP / 5xx retries to recover
  from network blips. New tests cover all four retry-state-machine
  branches.
- **`Config::warn_on_inline_secrets(path)`** — `Config::load_from_path`
  emits a `tracing::warn!` when `[third_party] api_key` is set inline,
  recommending `CAPTCHAFORGE_THIRDPARTY_API_KEY` env var instead.
  Stops accidental secret commits to git alongside config.
- **`PRODUCTION.md` deployment guide** — 200+ lines covering
  pre-launch sanity checks, the three configuration layers, telemetry
  sink choice + alert thresholds, token-cache lifecycle gotchas,
  pattern-store persistence patterns, third-party adapter operational
  notes, browser lifecycle / crash recovery, security guidance, and
  upgrade procedure.

#### Changed
- **Bench `solve` / `accuracy` / `throughput` suites rewired to
  `Config::default().build_chain()`.** Previously instantiated
  `BehavioralCaptchaSolver` etc. directly, bypassing the cache,
  provider registry, third-party solver, and telemetry — so the bench
  measured the individual solvers, not the production pipeline.
  Now reports the chain's actual winning method per fixture; cache
  short-circuit is part of the measured path. The `solver` column is
  the chain's `r.method` instead of a hand-coded label.
- **Bench `--suite real-waf` (CLI variant) + `--with-network` flag**
  added so `--suite all --with-network` runs the network-dependent
  proof end-to-end. Without `--with-network`, `--suite all` skips
  `real_waf` so CI offline doesn't fail.
- **Default fixture count: 34 → 37** (added 3 real-WAF fixtures).
  `fixtures_count_is_34` test renamed to `fixtures_count_is_37`.

#### Fixed
- **CLI `cmd_solve` no longer duplicates chain construction** —
  delegates to `Config::build_chain()`, single source of truth shared
  with `auto_solve`.

### Tier-A discovery, cache-cookies bridge, fail-fast third-party (2026-05-11, round 4)

#### Added
- **`Config::build_chain()`** — single canonical chain constructor: chain config + token cache + provider registry + Behavioral + VLM (env/TOML-aware) + Audio + Third-party. Used by both `auto_solve()` and the CLI's `solve` subcommand so they share the same wiring.
- **`TokenCache::put_full(.., cookies)`** + `CachedToken::cookies()`. Cache entries now carry both the solved token AND the cookies captured at the original solve. Cache hits replay the trusted session via `CaptchaSolveResult::cookies` so the next request rides the WAF/vendor's session — without this a hit returned only the token and the next navigation re-triggered the captcha. Back-compat `put` / `put_with_ttl` continue to work (cookies field stays empty).
- **Terminal-error fail-fast in `ThirdPartyCaptchaSolver::poll_result`.** Maps the documented 2captcha error codes (`ERROR_KEY_DOES_NOT_EXIST`, `ERROR_ZERO_BALANCE`, `IP_BANNED`, `ERROR_GOOGLEKEY`, `ERROR_CAPTCHA_UNSOLVABLE`, etc.) to immediate `Err` instead of polling 30 more times — saves up to 150s of wall-clock per doomed task and avoids submitting-without-results for paid services.

#### Changed
- **`captchaforge::auto_solve(page)` discovers Tier-A config.** Calls `Config::discover()` and builds the chain via `build_chain()`, so a user's `.captchaforge.toml` reaches the one-call API automatically. With no config file present, `discover()` returns defaults — the no-config install gets the same chain shape as before.
- **CLI `cmd_solve` builds the chain via `Config::build_chain()`** instead of duplicating the wiring logic. Single source of truth for chain construction across binaries.
- **Chain success path stores cookies in cache** via `cache.put_full(.., r.cookies.clone())` so the cache-hit replay path actually has cookies to replay.
- **`solver/mod.rs` docstring + `ARCHITECTURE.md` ASCII diagram** refreshed to show the four-solver chain (Behavioral / VLM / Audio / Third-party / Human-fallback) and the Tier-A config layer.

### Third-party adapters, EMA pattern store, env + Tier-A config (2026-05-11, round 3)

#### Added
- **`ThirdPartyCaptchaSolver` + `ThirdPartyService`.** Implements `SolveMethod::ThirdPartyService` against the 2captcha-compatible HTTP protocol used by 2captcha, CapMonster, and CapSolver — every TOML-bundled vendor that recommends `ThirdPartyService` now actually has a solver to land on. Constructors: `two_captcha()`, `cap_monster()`, `cap_solver()`, `custom_endpoint(base_url)`. API key via `with_api_key(...)` or `CAPTCHAFORGE_THIRDPARTY_API_KEY` env var. Wired into `CaptchaSolverChain::default_chain()` as the fourth solver; `supports()` returns false without an API key, so a no-key install sees no behaviour change. Supported shapes: Turnstile, reCAPTCHA v2/v3, hCaptcha, DataDome, Arkose/FunCaptcha, GeeTest v3+v4 (AWS WAF needs `iv`+`context` not in the standard `CaptchaInfo` so it is intentionally skipped). On success injects the token into the page (`cf-turnstile-response` / `g-recaptcha-response` / `h-captcha-response`) and captures session cookies.
- **`captchaforge::config::Config` (Tier-A operational config).** TOML-driven config layer for timeouts, cache TTL, VLM endpoint/model, third-party service selection. Compiled defaults → `.captchaforge.toml` → CLI flags. `Config::discover()` walks `$CAPTCHAFORGE_CONFIG` / `./.captchaforge.toml` / `./captchaforge.toml` / `$XDG_CONFIG_HOME/captchaforge/config.toml`. Materialisers: `chain_config()`, `solve_config()`, `build_token_cache()`, `build_vlm_solver()`, `build_third_party_solver()`. `deny_unknown_fields` so typos in keys surface as parse errors instead of silent ignores.
- **CLI `--config <file>` flag.** Override the discovery search path explicitly; otherwise discovery runs. `cmd_solve` now builds the chain via the loaded config (cache TTL, VLM endpoint, third-party service all reachable from the shell).
- **`.captchaforge.toml.example`.** Bundled schema reference with every Tier-A field commented + defaults inline.
- **`CAPTCHAFORGE_VLM_ENDPOINT` + `CAPTCHAFORGE_VLM_MODEL` env vars.** `VlmCaptchaSolver::new()` reads them once at construction time. Builder methods (`with_endpoint` / `with_model`) still beat env. Empty env values fall through to compiled defaults. New `pub(crate) new_with_env(closure)` overload makes precedence testable without `unsafe` env mutation (the crate is `#![forbid(unsafe_code)]`).
- **`TokenCache::ttl()` accessor.** Exposes the configured default TTL so the config layer can verify it round-trips and tests can assert the wiring.

#### Changed
- **`PatternStore` switched from cumulative-mean to bounded-α EMA.** Old `α = 1/n` made vendor migrations register slowly: 100 historical wins drowned out 30 fresh failures, so the chain kept routing to a method that no longer worked. New behaviour: `α = 1/n` for the first 10 samples (bootstrap matches old semantics), then `α = 0.2` so a flipped vendor surfaces as the new winner within ~10 alternating samples. `CaptchaPattern` now tracks per-method stats in a `methods: HashMap<String, MethodStat>` field; `best_method` is the per-method EMA winner with ties broken by faster `avg_solve_time`. Ships with regression tests that prove (a) bootstrap matches cumulative-mean, (b) EMA flips winner within 10 samples after a vendor migration, (c) ties resolve to the faster method.
- **Built-in solvers' `supports()` accept TOML-rule (`Custom(_)`) kinds.** Behavioral / VLM can attempt any visible challenge — without this opt-in the provider-driven routing for `Custom` had nothing to land on once `ordered_solvers_for_custom` was tightened to honour `supports()`. Audio stays narrow (RecaptchaV2 + AudioCaptcha only) since it depends on the accessibility-audio button.
- **`ordered_solvers_for_custom` now filters on `supports(kind)`.** Previously matched purely on method name, which would route Custom captchas to the third-party solver even without an API key — wasting one round-trip per attempt. The combined filter is now `s.method() == recommended && s.supports(kind)`.
- **CLI `cmd_solve` builds the chain via the loaded config** instead of `default_chain()`, so cache TTL / VLM endpoint / third-party service overrides reach the runtime.

#### Fixed
- **Default chain previously had three solvers with the protocol-recommended `ThirdPartyService` method matching nothing.** Adding `ThirdPartyCaptchaSolver` plugs the gap; `default_chain_has_four_solvers_in_documented_order` pins the order.

### Cookies, polish, local proof (2026-05-11, round 2)

#### Added
- **`captchaforge::cookies` module + cookie capture.** All three built-in solvers (Behavioral, VLM, Audio) now capture session cookies on successful solve and surface them in `CaptchaSolveResult.cookies`. Replay via `cookies::apply_to_page(&page, &result.cookies)` to preserve the WAF/vendor's trusted session across navigations and avoid re-triggering the captcha. New `CapturedCookie` struct with `is_expired_now` / `keep_persistent_alive` helpers.
- **`examples/architecture_proof.rs`** — local-runnable markdown report covering TokenCache hit/miss latency, ProviderRegistry::find_by_kind latency, RuleDetector probe generation cost, and per-rule HTML fixture match latency. Browser-free, runs in well under a second after a release build. Use to spot hot-path regressions; the round-1 numbers were 55ns / 27ns / 12ns / 0.76µs / 0.02–0.07µs per match.

#### Changed
- **README "Strategies" section + ARCHITECTURE.md ASCII diagram** updated to reflect the post-sweep architecture (cache layer, provider routing, telemetry, cookie capture).

### Added
- Property-based testing: 40+ proptest properties across detect, solver, frame, and behavior modules
- Snapshot testing: insta-powered JSON snapshots for all key data structures
- Fuzz testing: 4 fuzz targets (fuzz_detect, fuzz_solve_config, fuzz_selector, fuzz_pattern_key)
- Chaos tests: Resilience testing for extreme inputs, NaN/Inf handling, empty strings
- GitHub Actions CI/CD: ci.yml, coverage.yml, bench.yml, fuzz.yml
- Comprehensive documentation: ARCHITECTURE.md, TESTING.md, BENCHMARKING.md, CONTRIBUTING.md, CHANGELOG.md
- Benchmark suite expansion: 35+ self-hosted CAPTCHA fixtures, 10 benchmark suites, 6 output formats

### Changed
- Test organization refactored into 5-tier pyramid: unit -> property -> snapshot -> integration -> chaos

## [0.2.1] - 2025-05-09

### Added
- CaptchaSolveResult.screenshot field for human-in-the-loop review
- VlmCaptchaSolver.with_endpoint() and .with_model() builder methods
- SolveConfig struct with all formerly-hardcoded timeouts
- CDP frame evaluation module (src/frame.rs) for cross-origin iframe piercing
- verify_token_in_frames() to prevent false-positive success claims
- BehavioralCaptchaSolver now verifies tokens before returning success
- AudioCaptchaSolver now verifies g-recaptcha-response post-submission
- ImageCaptcha and AudioCaptcha detection arms in detect.js
- 8 real-browser integration tests

### Fixed
- Removed 6 credential-leaking println! statements from scanner hot path
- Replaced naive split("://") with url::Url parsing in extract_domain
- Fixed lock poisoning in PatternStore with unwrap_or_else(|e| e.into_inner())
- Turnstile, Audio, reCAPTCHA v3 now verify tokens via verify_token_in_frames()

## [0.2.0] - 2025-05-08

### Added
- CaptchaSolverChain with ordered_solvers() and PatternStore historical success tracking
- BehavioralCaptchaSolver with Bezier mouse curves and realistic timing
- VlmCaptchaSolver with Ollama-compatible multimodal LLM integration
- AudioCaptchaSolver with STT endpoint support
- CaptchaPattern and PatternStore for per-domain success memory

### Changed
- Detection engine expanded to support hCaptcha, generic image/audio CAPTCHAs

## [0.1.0] - 2025-05-07

### Added
- Initial release: Turnstile and reCAPTCHA v2/v3 detection
- Basic CaptchaSolver trait
- captcha_detect back-compat re-export