Expand description
Bounded concurrency budget for the multi-thread one-shot runtime.
§Workload classification (Rules Rust — parallelism)
- Class: mixed — I/O-bound HTTPS on the Tokio multi-thread runtime plus
CPU-bound HTML/JSON parse on the
spawn_blockingpool. - Stages: (1) network/cache GET on async workers → (2) body decode + scrape/markdown on blocking workers gated by this budget → (3) emit.
- Bound: every fan-out of CPU work acquires a permit from
ConcurrencyBudget’sArc<Semaphore>. Drop of the permit (RAII) returns capacity. HTTP remains one in-flight request per command today; the same budget is ready for multi-GET fan-out if a batch command is added. - Formula (auto):
min(cpus, floor((free_ram_mib / 2) / RAM_PER_TASK_MIB))clamped to[1, MAX_AUTO_CONCURRENCY]. Free RAM usesMemAvailableon Linux (/proc/meminfo); other platforms fall back to CPU count only. - RSS ground truth:
RAM_PER_TASK_MIBrounds up a single HTML parse of a body nearHARD_MAX_BODY_BYTES(10 MiB body + DOM + markdown ≈ 2–3× peak). Re-measure with/usr/bin/time -v docsrs-cli search-in-crate …when the body cap or scraper stack changes materially. - Override: CLI
--max-concurrency=Normax_concurrencyin XDGconfig.tomlonly (not productDOCSRS_CLI_*env).0/ omit = auto.
Structs§
- Concurrency
Budget - Shared admission gate for CPU-bound (and future multi-GET) work.
Constants§
- MAX_
AUTO_ CONCURRENCY - Hard ceiling for auto-computed concurrency (CLI, not a server farm).
- MAX_
EXPLICIT_ CONCURRENCY - Hard ceiling for explicit
--max-concurrency/ config override. - RAM_
PER_ TASK_ MIB - Estimated peak RSS (MiB) for one CPU-bound parse task.
Functions§
- auto_
max_ concurrency - Auto permits from CPUs and free RAM (50% free-RAM safety margin).
- max_
blocking_ threads - Cap for Tokio’s
spawn_blockingpool (Rules Rust — rede / paralelismo). - resolve_
max_ concurrency - Resolve permit count from config override (
0= auto). - runtime_
worker_ threads - Worker thread count for the multi-thread Tokio runtime (I/O + signal tasks).