Expand description
Concurrent execution of multiple scenario files across isolated browsers. Concurrent execution of multiple scenario files.
A single llm-browser-testkit run a.toml b.toml c.toml invocation runs
each file on its own isolated browser (separate Chrome process, so
cookies, localStorage, and other session state can never leak between
files). Each file’s own tests always run sequentially on that file’s
browser.
§Concurrency control
[config] concurrency_group = "name"makes every file that declares the same group mutually exclusive: they never execute at the same time. A file without a group gets its own implicit group, so distinct files run in parallel by default.- Concurrency is bounded by a
ParallelMode: an exact manual count (--parallel N) or an auto mode that adapts to the machine.
§Auto scaling
In auto mode the orchestrator never assumes a fixed per-browser memory footprint. It instead learns it by trial and error:
- it probes available system memory once and keeps a running estimate of
how many bytes one browser actually holds (
available_before−available_afteraround each run), clamped to sane bounds; - after each successful file it raises the concurrency limit toward
capacity = available / learned_footprint, and otherwise ramps up one at a time; - it guards launches: a worker will not start a new browser while
less than an absolute [
MIN_HEADROOM_BYTES] is free, or there is no room for one more browser of the learned footprint. It never uses a fraction oftotalmemory, so it stays correct in VMs/containers wheretotalreports the host’s much larger RAM; - when a launch fails with an out-of-memory-style error it halves the
limit and retries the file with backoff, up to
[
MAX_LAUNCH_RETRIES].
Bounds come from --parallel-min / --parallel-max (default 1 /
unlimited). When system memory cannot be measured the auto ceiling falls
back to a conservative [DEFAULT_AUTO_CEILING].
Structs§
- Memory
Info - Best-effort snapshot of total / available physical memory.
- Parallel
Run - Result of a parallel batch run: the merged report plus combined usage.
- RunOptions
- Options controlling a parallel batch run.
- Scenario
File - Runs one scenario file, ready for the scheduler. CLI overrides applied and viewport matrix expanded by the caller.
Enums§
- Parallel
Mode - How concurrency is chosen for a batch of files.
Functions§
- mode_
from_ cli - Builds the
ParallelModefrom CLI flags.--parallel Nselects manual mode; otherwise auto mode with the given min/max bounds. - probe_
memory_ async - Best-effort probe of total/available physical memory (Linux then macOS).
Returns
Nonewhen unavailable; the scheduler then relies on trial and error. - run_
scenarios - Runs every file concurrently, respecting the concurrency mode and
per-file groups, each on its own isolated browser. Emits one
RunStarted/RunFinishedpair for the batch and merges cost/usage.