# Work Log
Tracks the ongoing improvement programme: bug fixes, API design, refactoring,
testing, and performance work. Each section records what was done and why.
### Follow-up note — feature boundary tightening (2026-03-11)
Tightened the internal feature boundary so non-emulation users do not compile
the full preset/profile layer by default.
- `browser_emulation` presets and profile-builder types are now behind the
`emulation` feature.
- Non-`emulation` builds keep only a lightweight internal placeholder profile
so shared protocol/pool code can compile without carrying the preset model.
- `btls`-specific fingerprint structs are now compiled only when `btls-backend`
or `emulation` is enabled.
- `ClientBuilder` / `RequestBuilder` profile resolution paths are now no-ops
when `emulation` is disabled.
- Verified feature combinations:
- `cargo check`
- `cargo check --features btls-backend`
- `cargo check --features emulation`
### Follow-up note — TLS planning tests and capability matrix (2026-03-11)
Filled two of the main remaining maintenance gaps after the structural rewrite:
- Added an explicit internal `btls` TLS build-plan layer so the mapping from
`TlsConfig` to `btls` builder state is testable without a live handshake.
- Added offline golden-style tests for:
- ALPN wire-format encoding
- Boring TLS version parsing
- the effective `btls` application plan for a custom fingerprint
- builder-side propagation of verify mode and TLS version bounds
- Added a backend capability matrix to the main design document so feature
combinations and promised behavior are visible to maintainers.
### Follow-up note — live ClientHello capture tests (2026-03-11)
Extended the `btls` validation layer from offline planning tests to real
ClientHello capture tests.
- Added a raw TCP capture harness that reads the outbound TLS handshake bytes
without requiring a full server-side TLS stack.
- Added live assertions for:
- HTTP/1-safe ALPN emission on the `btls` path
- custom supported-groups emission from a `BoringTlsFingerprint`
- preset-dependent signature-algorithm lists (`Chrome136` vs `Firefox128`)
- This upgrades the remaining TLS validation gap from "no live coverage" to
"no full fixture corpus / broader preset corpus yet".
### Follow-up note — preset ClientHello fixture corpus (2026-03-11)
Extended live TLS validation from isolated field checks to normalized preset
fixtures for the first emulation set.
- Added a normalized ClientHello fixture assertion layer that strips GREASE and
other unstable elements while preserving stable fingerprint features.
- Added fixture coverage for:
- `Chrome136`
- `Firefox128`
- `Safari18_4`
- Each preset fixture now verifies:
- ALPN
- supported versions
- supported groups
- cipher-suite ordering (normalized)
- signature-algorithm ordering
- presence/absence of key extensions such as ECH grease and certificate
compression
### Follow-up note — normalized byte-level TLS corpus (2026-03-11)
Extended the live TLS fixture layer from field-level assertions to canonical
byte-level snapshots for the current emulation presets.
- Added a ClientHello normalization pass that stabilizes:
- random bytes
- session IDs
- GREASE values
- key-share key material
- ECH grease payload contents
- optional padding extension churn
- Added normalized byte-level snapshot fixtures for:
- `Chrome136`
- `Firefox128`
- `Safari18_4`
- The snapshot corpus keeps the wire-shape fidelity that matters for
regression detection while intentionally masking bytes that are expected to
change on every handshake.
### Follow-up note — HTTP/2 lifecycle probes (2026-03-11)
Extended HTTP/2 validation from payload/header unit tests to connection-start
wire probes.
- Added an h2 TLS probe server that captures:
- client SETTINGS payload
- stream 0 WINDOW_UPDATE
- PRIORITY frames
- the decoded first request header order
- Added lifecycle assertions for:
- `Chrome136` emulation startup on the real h2 path
- a custom `Http2Fingerprint` with non-default SETTINGS order,
`initial_connection_window_size`, and `PRIORITY` emission
- This closes the gap between "the frame builder can encode the right bytes"
and "the client actually emits the expected startup frame sequence on a live
h2 connection".
### Follow-up note — online emulation smoke suite (2026-03-11)
Added an opt-in external smoke suite for the current preset set.
- New integration test: `tests/emulation_online_smoke.rs`
- Current probe target defaults to `https://tls.peet.ws/api/all`
- Supported override:
- `UGI_EMULATION_SMOKE_URL`
- Safety gate:
- tests are `#[ignore]`
- `UGI_RUN_ONLINE_SMOKE=1` is required
- Verified manually against the public probe for:
- `Chrome136`
- `Firefox128`
- `Safari18_4`
- Assertions cover:
- negotiated HTTP version
- user-agent shape
- ALPN seen by the probe
- normalized supported-groups shape
- HTTP/2 SETTINGS order and values
- connection-level WINDOW_UPDATE increment
- first-request header order
### Follow-up note — HTTP/1 original field-name serialization (2026-03-11)
Closed the remaining HTTP/1 emulation gap around wire-format header names and
generated-header ordering.
- Refactored the HTTP/1 encoder so generated headers such as `Host`,
`Content-Length`, `Accept-Encoding`, `Cookie`, and proxy auth participate in
the same profile-driven ordering pass as caller-provided headers.
- Wired `Http1Fingerprint.original_header_case` into serialization so selected
headers are emitted with configured field-name casing even though the
internal header map still normalizes names to lowercase.
- Updated the built-in browser presets to include a default configured casing
table for common HTTP/1 headers.
- Added regression tests covering:
- generated `Host` placement inside the configured header order
- configured field-name serialization for both caller-provided and generated
headers
### Follow-up note — unified emulation entrypoint (2026-03-11)
Collapsed the preferred public builder API down to a single `.emulation(..)`
entrypoint for both presets and custom profiles.
- `ClientBuilder::emulation(..)` and `RequestBuilder::emulation(..)` now accept
either a versioned `Emulation` preset or a custom `EmulationProfile`.
- Added `From<Emulation> for EmulationProfile` so preset and profile callers go
through the same configuration path.
- Kept `.browser_profile(..)` and `.emulation_profile(..)` as compatibility
aliases for now, but they now forward to `.emulation(..)` instead of being
separate primary APIs.
- Added regression tests proving that custom profiles passed through
`.emulation(..)` still apply default headers and HTTP/2 preference behavior.
### Follow-up note — emulation examples and integration coverage (2026-03-11)
Rounded out the end-user surface around emulation so the public API is now
demonstrated and exercised outside unit tests.
- Added feature-gated examples:
- `examples/emulation_preset.rs`
- `examples/emulation_custom.rs`
- Added `tests/emulation_integration.rs` to cover:
- preset default headers applied end-to-end
- request-level `.emulation(profile)` replacing the client-level preset
without leaking client defaults into the effective request
- Updated `README.md` so the Cargo feature table, public API overview, and
example commands all reflect the unified `.emulation(..)` entrypoint.
### Follow-up note — proxy tunnel integration coverage and protocol doc sync (2026-03-11)
Closed one of the lingering protocol-readiness gaps and synchronized the stale
progress docs with the actual codebase state.
- Added `tests/proxy_tunnel_integration.rs` with end-to-end HTTPS tunnel
coverage for:
- HTTP CONNECT without auth
- HTTP CONNECT with auth
- SOCKS5 without auth
- SOCKS5 with auth
- While landing that suite, found and fixed a real protocol bug:
- HTTP CONNECT requests were missing the `HTTP/1.1` token in the request line
- fixed by introducing a shared CONNECT request builder used by both the
HTTP/1 and HTTP/2 proxy tunnel paths
- Updated `docs/protocol-readiness-progress.md` so it no longer lists already
fixed items such as:
- WebSocket async DNS
- WebSocket masking randomness
- `base64_encode` duplication
- H2 write-failure teardown
- keepalive config pool-key splitting
- Marked the old Phase 4 "remaining from original plan" checklist as closed
where coverage already exists or is now satisfied.
### Follow-up note — HTTP/2 HPACK adaptive encoder (2026-03-12)
Closed the remaining HTTP/2 header-compression gap around peer
`SETTINGS_HEADER_TABLE_SIZE`.
- Replaced the old `HeaderCodec` binary switch ("fully indexed" vs
"fully non-indexed") with a stateful HPACK encoder maintained inside ugi.
- The new encoder now:
- tracks the peer's effective header-table limit
- emits HPACK dynamic table size updates when that limit changes
- preserves dynamic-table reuse for entries that still fit
- safely avoids reuse for oversized entries that cannot remain resident
- Updated the HTTP/2 connection path to feed the actual peer table size into
request header encoding instead of reducing the behavior to a boolean.
- Added focused regressions covering:
- reuse with a small peer table
- table-size update emission after a peer settings change
- non-reuse of entries larger than the peer table
- Synchronized `docs/protocol-readiness-progress.md` so HPACK adaptation is no
longer listed as an open issue.
### Follow-up note — HTTP/3 GOAWAY pool-slot eviction (2026-03-12)
Closed a lifecycle gap where an HTTP/3 connection that had already received
`GOAWAY` could remain parked in the pool after becoming idle.
- Updated the H3 pool eviction rules so a quiescent connection with
`accepting_new_requests = false` is treated as immediately evictable instead
of waiting for idle-timeout expiry.
- Marked GOAWAY'd H3 connections as `close_when_idle`, so the background task
exits promptly once in-flight work drains.
- Added a focused regression,
`evicts_idle_http3_goaway_connection_before_reusing_pool_slot`, that runs
with `max_idle_per_host(1)` and verifies that a stale GOAWAY connection does
not consume the only reusable slot and force a third QUIC handshake.
- Synchronized `docs/protocol-readiness-progress.md` to record this as another
resolved HTTP/3 lifecycle fix while leaving broader H3 hardening work open.
### Follow-up note — HTTP/3 gRPC binary metadata coverage (2026-03-12)
Extended the gRPC interop coverage so HTTP/3 now exercises the same binary
metadata helper path that HTTP/2 already had.
- Added `grpc_http3_binary_metadata_round_trips_through_helpers` to the H3
integration suite.
- The new regression verifies:
- outbound `metadata_bin(...)` request encoding
- inbound header binary metadata decoding
- `grpc-status-details-bin` parsing from H3 trailers
- trailer binary metadata access via the existing response helpers
- Updated `docs/protocol-readiness-progress.md` so the gRPC section reflects
that unary binary metadata round-trip coverage now exists for both H2 and H3.
### Follow-up note — HTTP/3 gRPC protobuf byte-path coverage (2026-03-12)
Extended the H3 gRPC matrix so the raw protobuf byte path is no longer only
covered on HTTP/2.
- Added `executes_grpc_protobuf_bytes_request_over_http3`.
- The new regression verifies:
- `GrpcCodec::Protobuf` selects `application/grpc+proto` over H3
- raw `message_bytes(...)` requests are framed correctly
- unary protobuf-byte responses can be read back through
`response.message_bytes()`
- Updated `docs/protocol-readiness-progress.md` so the gRPC section and
regression list reflect that unary protobuf-byte coverage now exists on both
HTTP/2 and HTTP/3.
### Follow-up note — HTTP/3 GOAWAY stale classification (2026-03-12)
Tightened the H3 GOAWAY semantics so unsent work is no longer surfaced as a
generic transport failure.
- Added explicit helpers for:
- pending/opening request GOAWAY failures
- stream-ID rejection against the advertised GOAWAY identifier
- stale errors for rejected in-flight streams
- Wired those helpers into the H3 connection task so:
- queued requests that never left the client are failed as
`ErrorKind::StaleConnection`
- active streams at or above the GOAWAY identifier are also treated as stale
- Added focused unit regressions in `src/protocol/http3.rs` that lock the
stream-ID threshold and stale-error classification semantics in place.
- Re-ran the H3 regression suite to confirm this tighter classification does
not break existing GOAWAY retry or pool-eviction behavior.
### Follow-up note — HTTP/3 peer-close stale classification (2026-03-12)
Closed another lifecycle gap around reused HTTP/3 connections that die before
the next response begins.
- Tightened the H3 connection-close path so when QUIC reports the connection as
closed by peer:
- pending requests are failed as `ErrorKind::StaleConnection`
- opening requests that had not obtained a stream ID yet are also failed as
stale
- in-flight streams that still have not received final response headers are
failed as stale rather than generic transport errors
- This preserves transparent retries for safe requests on reused H3
connections when the peer closes the connection before the next response has
started, while still leaving partially-started responses on the transport
error path.
- Extended the H3 script server so tests can close a QUIC connection before
sending any response bytes for a request.
- Added focused regressions:
- `retries_http3_request_after_peer_closes_reused_connection_before_response`
- `protocol::http3::tests::peer_close_unsent_request_errors_are_stale`
- `protocol::http3::tests::peer_close_before_response_errors_are_stale`
- Re-validated the existing GOAWAY retry and idle-eviction regressions to make
sure the new peer-close classification does not disturb the earlier H3
lifecycle fixes.
### Follow-up note — HTTP/3 gRPC client-streaming unary coverage (2026-03-12)
Extended the H3 gRPC matrix so client-streaming unary requests are no longer
only covered on HTTP/2.
- Added `executes_grpc_client_streaming_json_request_over_http3`.
- The new regression verifies:
- `GrpcRequestBuilder::messages(...)` frames multiple JSON messages correctly
over HTTP/3
- the request stays streaming/chunked at the transport layer (no
`content-length`)
- the unary H3 response still decodes back through the existing gRPC helper
path
- Updated `docs/protocol-readiness-progress.md` so the gRPC coverage summary
reflects that client-streaming unary JSON is now covered on both H2 and H3.
### Follow-up note — gRPC client-streaming protobuf-byte coverage (2026-03-12)
Extended the raw protobuf streaming path so `messages_bytes(...)` is no longer
only exercised through unary or duplex helpers.
- Added:
- `executes_grpc_client_streaming_protobuf_bytes_request_over_http2`
- `executes_grpc_client_streaming_protobuf_bytes_request_over_http3`
- The new regressions verify:
- `GrpcRequestBuilder::messages_bytes(...)` frames multiple raw protobuf
payloads correctly
- both H2 and H3 transports keep the request streaming without a
`content-length`
- unary protobuf-byte responses still decode back through
`response.message_bytes()`
- Updated `docs/protocol-readiness-progress.md` so the gRPC coverage summary
records client-streaming protobuf-byte coverage on both H2 and H3.
### Follow-up note — HTTP/3 gRPC error-path coverage (2026-03-12)
Filled two more H3 gRPC interop gaps around non-OK status handling.
- Added:
- `streaming_grpc_http3_response_surfaces_trailing_error_status`
- `exposes_grpc_error_status_from_trailers_only_response_over_http3`
- The new regressions verify:
- streaming H3 gRPC responses can yield a successful message and then surface
a non-zero `grpc-status` from trailers as a transport error while still
preserving decoded status/trailer metadata
- trailers-only H3 gRPC responses that place `grpc-status` and
`grpc-message` in the initial header block are exposed through the same
buffered response helpers as the existing H2 path
- Updated `docs/protocol-readiness-progress.md` so the gRPC coverage summary
now records trailers-only error coverage and trailing-error streaming
coverage on both H2 and H3.
### Follow-up note — HTTP/3 gRPC server-streaming coverage (2026-03-12)
Closed another gRPC/H3 interop gap by adding explicit server-streaming
response coverage.
- Added `reads_grpc_server_streaming_messages_over_http3`.
- The new regression verifies:
- a unary JSON request over H3 can read multiple gRPC response messages via
`response.messages::<T>()`
- the wire request still carries the expected path, content type, and unary
request body framing
- Updated `docs/protocol-readiness-progress.md` so server-streaming unary JSON
coverage is now recorded on both HTTP/2 and HTTP/3.
### Follow-up note — HTTP/2 duplex gRPC metadata parity (2026-03-12)
Closed a smaller but real interop asymmetry between the H2 and H3 duplex gRPC
tests.
- Extended `grpc_duplex_call_supports_interleaved_send_and_receive_over_http2`
to assert the same binary metadata accessors that were already covered on H3:
- `call.metadata_bin("x-stream")`
- `call.trailer_metadata_bin("x-trace")`
- This confirms the existing H2 bidi probe server's binary metadata and
trailer metadata are surfaced through the duplex helpers, not just emitted on
the wire.
- Updated `docs/protocol-readiness-progress.md` so duplex binary metadata
coverage is now described as present on both HTTP/2 and HTTP/3.
### Follow-up note — HTTP/3 mid-response retry boundary (2026-03-12)
Locked down the complementary half of the new H3 stale-classification work:
requests may retry transparently before a response starts, but not after.
- Extended the H3 script server so a test response can close the QUIC
connection immediately after emitting response body bytes.
- Added `does_not_retry_http3_request_after_response_headers_have_started`.
- The new regression verifies:
- a reused H3 connection can return response headers, then die mid-body
- the client surfaces that failure on the active response as a transport
error instead of silently replaying the request
- the next explicit request still recovers on a fresh QUIC connection
- Updated `docs/protocol-readiness-progress.md` so the HTTP/3 lifecycle
section now records the retry boundary explicitly: stale retry stops once a
response has started.
### Follow-up note — HTTP/2 placeholder priority-tree support (2026-03-12)
Extended the H2 emulation model so startup PRIORITY frames are no longer
limited to the current request stream.
- Added `Http2PrioritySpec.stream_id: Option<u32>`.
- `None` keeps the old behavior and targets the current request stream.
- `Some(id)` emits a PRIORITY frame for that explicit stream ID, which is
useful for placeholder-tree startup fingerprints.
- Wired the H2 connection task to honor the explicit target stream ID while
still rejecting invalid stream `0`.
- Added `http2_custom_fingerprint_can_emit_placeholder_priority_tree_before_headers`
to verify multiple placeholder PRIORITY frames are emitted before the first
request HEADERS frame and are captured with the expected target stream IDs.
- Updated `docs/browser_emulation.md` so the H2 lifecycle checklist now marks
this area as partially covered instead of entirely missing.
### Follow-up note — HTTP/2 phased priority-frame support (2026-03-12)
Extended the H2 lifecycle model so priority fingerprints are no longer limited
to "all before HEADERS".
- Added `Http2PriorityPhase` with:
- `BeforeHeaders`
- `AfterHeaders`
- Wired the H2 connection task to emit fingerprinted PRIORITY frames in two
passes around the request HEADERS write, while still preserving explicit
stream targeting and rejecting invalid stream `0`.
- Added
`http2_custom_fingerprint_can_emit_post_headers_reprioritization` to verify
a custom fingerprint can emit:
- a placeholder-tree PRIORITY frame before the first request HEADERS
- a request-stream reprioritization frame after HEADERS
- Updated `docs/browser_emulation.md` so the "richer H2 lifecycle fingerprint
behavior" item now reflects that both pre-HEADERS and post-HEADERS priority
choreography are covered, while broader browser-specific frame sequences
remain open.
### Follow-up note — QUIC / HTTP/3 fingerprint design draft (2026-03-12)
Reduced one of the remaining browser-emulation documentation gaps by turning
the QUIC/H3 item from a placeholder into an explicit design draft.
- Updated `docs/browser_emulation.md` to match the current code reality:
- `Http3Fingerprint` already exists, but is intentionally minimal today
- Added a concrete future split between:
- `QuicFingerprint` for transport parameters
- `Http3Fingerprint` for HTTP/3 SETTINGS / QPACK behavior
- Recorded minimum design rules for any future browser-grade QUIC claim:
- pool isolation must hash the full QUIC fingerprint
- QUIC transport and H3 application fingerprints must be modeled separately
- browser-grade claims require live QUIC/H3 probes, not only unit coverage
- Updated the coverage checklist so browser-grade QUIC design is now marked as
partially covered rather than entirely missing.
### Follow-up note — HTTP/3 idle peer-close pool coverage (2026-03-12)
Added the missing regression for a quieter H3 closure path that was not yet
explicitly covered.
- Added `evicts_idle_http3_peer_closed_connection_before_reusing_pool_slot`.
- The new regression verifies that when a response completes successfully and
the peer then closes the QUIC connection, the stale idle connection does not
occupy the only `max_idle_per_host(1)` slot and force an unnecessary third
handshake.
- This complements the earlier GOAWAY pool-slot eviction test with the
corresponding peer-close path.
### Follow-up note — HTTP/3 idle peer-close recovery (2026-03-12)
Covered the adjacent H3 lifecycle case where the peer closes the connection
while it is idle between requests rather than during an active request.
- Added a new H3 test-server hook,
`close_connection_after_response_delay(...)`, so the script server can close
a QUIC connection after a completed response and a short idle delay.
- Added `opens_new_http3_connection_after_idle_peer_close`.
- The new regression verifies:
- the first request completes successfully
- the peer closes the now-idle QUIC connection in the background
- the next request opens a fresh H3 connection instead of trying to reuse a
stale pooled handle
- Updated `docs/protocol-readiness-progress.md` so idle peer-close recovery is
recorded explicitly in the HTTP/3 lifecycle section.
### Follow-up note — proxy tunnel HTTP/2 integration coverage (2026-03-12)
Expanded the proxy interop suite beyond the HTTP/1.1 request path.
- Added end-to-end HTTPS tunnel coverage for HTTP/2 in
`tests/proxy_tunnel_integration.rs`:
- `http_connect_proxy_tunnels_https_over_http2`
- `socks5_proxy_tunnels_https_over_http2`
- The new tests terminate the tunnel on a real TLS+ALPN+h2 server rather than
only checking the CONNECT/SOCKS handshake in isolation.
- While adding that coverage, the h2 test harness needed a small fix:
- after sending the response, it now keeps polling the h2 connection briefly
so queued response frames are actually flushed before the server task exits
- Updated `docs/protocol-readiness-progress.md` so proxy coverage now records
HTTP/2 tunnel traffic in addition to the earlier HTTP/1.1 path.
### Follow-up note — HTTP/3 upload-stream error isolation (2026-03-12)
Closed the remaining local-request lifecycle hole in the H3 connection task.
- `src/protocol/http3.rs` now treats request-body producer errors as per-stream
failures inside `flush_outbound_data()` instead of bubbling them out as a
connection-wide task error.
- The affected HTTP/3 stream is now shut down with `H3_REQUEST_CANCELLED`,
the original transport error is surfaced back to that request, and the QUIC
connection remains reusable for subsequent requests.
- Added `request_body_stream_error_fails_only_the_failed_http3_stream` in
`src/protocol/http1.rs`, covering:
- first request body stream yields a local error after an initial chunk
- the failed request returns that transport error
- a second request succeeds on the same HTTP/3 connection
- Tightened the H3 test harness so temporary cert/key asset names are unique
across concurrent `cargo test` processes by including the process ID in the
temp-file path.
- Updated `docs/protocol-readiness-progress.md` so HTTP/3 lifecycle hardening
is now treated as complete for the current production-readiness scope, with
gRPC breadth and broader proxy / external interop remaining as the next
protocol-facing tasks.
### Follow-up note — GraphQL transport examples and 0.2.1 release prep (2026-03-12)
Added the last user-facing examples needed before the `0.2.1` cut.
- Added `examples/graphql_json.rs`:
- demonstrates the plain `serde` envelope pattern for GraphQL POST requests
- keeps GraphQL semantics outside the core `ugi` API surface
- Added `examples/graphql_graphql_client.rs`:
- demonstrates using `graphql_client::QueryBody` as the serialized request
body while keeping `ugi` as the HTTP transport
- Updated `README.md`:
- bumped dependency snippets to `0.2.1`
- documented the intended GraphQL integration model
- linked the new examples by name
- Bumped the crate version in `Cargo.toml` to `0.2.1`.