1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
//! Shared limits for the suites that start a **foreign engine** container.
//!
//! # The failure this exists to stop
//!
//! `interop_duckdb` and `interop_pyiceberg` start a container per test, and both
//! containers do real work before they print anything: DuckDB runs `INSTALL
//! iceberg`, and the Python one runs `pip install pyiceberg[pyarrow]`. Each is
//! seconds of network and CPU, and the wait strategy is a log line, so the clock
//! that matters is *time to first output* rather than time to start.
//!
//! Cargo runs the suite with one thread per core, so a dozen of those land at
//! once. Every one of them then competes for the same network and the same CPU,
//! every one takes longer than it would alone, and the ones that lose the race
//! trip `WaitContainer(StartupTimeout)`. The tell is that the failing *set*
//! changes from run to run, and that running either suite with
//! `--test-threads=1` passes every time.
//!
//! That is the worst kind of red suite: it is not reporting anything about the
//! code, and a suite that is red for reasons nobody can act on is one people
//! stop reading.
//!
//! # The fix is a bound, not a longer timeout
//!
//! A longer timeout alone only moves the threshold — the load still scales with
//! the core count of whatever machine runs it, so a bigger CI runner makes the
//! problem *worse*. [`engine_slot`] bounds how many foreign-engine containers
//! exist at once, which makes the per-container start time roughly independent
//! of the host. The generous [`STARTUP_TIMEOUT`] is then headroom for a slow
//! network rather than the mechanism.
//!
//! The bound is deliberately not 1. These suites are the slowest in the binary,
//! and serialising them entirely would add about a minute to every run for a
//! safety margin that [`MAX_CONCURRENT`] already provides.
use Duration;
use ;
/// How many foreign-engine containers may be starting or running at once.
///
/// Four keeps the suites parallel enough to be quick while staying far below the
/// point where they starve each other. Measured rather than guessed: at this
/// bound both suites pass repeatedly on a machine where the unbounded version
/// failed a different handful of tests on every run.
pub const MAX_CONCURRENT: usize = 4;
/// How long a foreign-engine container may take to print its first output.
///
/// Far above the library default, because the wait is not for a process to
/// start: it covers an `INSTALL`/`pip install` over the network. This is
/// headroom for a slow mirror, not the concurrency control — that is
/// [`engine_slot`].
pub const STARTUP_TIMEOUT: Duration = from_secs;
/// The gate itself.
static ENGINES: Semaphore = const_new;
/// Reserve one of the [`MAX_CONCURRENT`] foreign-engine slots.
///
/// Hold the returned permit for as long as the container is alive — dropping it
/// early would let the next test start a container while this one is still
/// competing for the network, which is the situation being avoided.
///
/// ```ignore
/// let _slot = engine_slot().await;
/// let container = GenericImage::new(..).start().await?;
/// // ... permit released when `_slot` drops, after `container` is done
/// ```
pub async