Crate loadpace 

Adaptive client-side load balancing for changing service fleets, with bounded backpressure.

A client may need to balance work across endpoints whose capacities and latencies differ or change as the fleet scales. Static rate and concurrency limits cannot adapt to those differences, while unbounded local queues hide overload. Loadpace learns an operating point for each endpoint, paces work toward it, predicts completion cost for endpoint selection, and stops local admission when the bounded scheduling horizon is full.

This crate contains the runtime-independent controller and deterministic simulator. Framework integrations live in separate crates such as loadpace-tower and loadpace-rama.

Deployment scope

Loadpace is intended for trusted microservice deployments where you control the clients sharing an endpoint and can deploy a compatible congestion controller to all of them. Its fairness mechanism assumes those clients cooperate: an unpaced or malicious client can take capacity from paced clients and invalidate the controller’s feedback assumptions.

Do not use Loadpace as the primary protection for a general-purpose public API. It is not an authorization mechanism, quota system, abuse-prevention boundary, or DDoS defense. Public traffic still needs server-enforced rate limits, quotas, and admission control.

Where to start

• Use EndpointController when integrating with a custom runtime or service abstraction.
• Use simulate to test controller settings against deterministic workloads before deploying them.
• Use the loadpace-tower or loadpace-rama adapter when your client is already built on one of those frameworks.

The repository also contains a tutorial, task-oriented how-to guides, a controller reference, and a design explanation.

Structs

ControllerSnapshot

A point-in-time view useful for metrics, tests, and P2C load prediction.

DispatchReservation

A request that has reserved a slot in the endpoint’s virtual queue.

EndpointConfig

Configuration for one adaptive endpoint.

EndpointController

The per-endpoint adaptive controller.

EndpointReport

Per-endpoint simulation results.

Gcra

A single-rate, spacing-oriented GCRA pacer.

Gradient2

A fractional Gradient2 operating-point controller.

Gradient2Config

Parameters for the fractional Gradient2 controller.

InFlightRequest

A request that has actually been dispatched and is now counted against the emergency inflight cap.

LatencyEstimator

Smoothed endpoint RTT estimates.

LatencyEstimatorConfig

Configuration for the endpoint latency estimator.

Probe

An active temporary probe.

ProbeSchedule

Randomized probe policy used internally by an crate::EndpointController.

SimulatedEndpoint

A fixed-service-time endpoint used by simulate.

SimulationConfig

Configuration for a deterministic offered-load simulation.

SimulationReport

Aggregate simulation results.

Enums

DispatchState

The result of asking whether a virtual queue reservation may dispatch.

Outcome

Classification supplied by the caller after a request completes.

ProbeKind

The temporary perturbations supported by Loadpace.

ScheduleError

A small error used by callers that want to model rejected scheduling explicitly in a simulator or their own adapter.

Functions

simulate

Runs a fixed offered-rate workload through adaptive endpoints and P2C.