Crate pingora_load_balancing 

Pingora Load Balancing utilities

This crate provides common service discovery, health check and load balancing algorithms for proxies to use.

Grouped selector internals

In LoadBalancerGroup<S>, S is the selector algorithm and its built data, such as a Ketama ring. The other types manage its configuration, rebuilding, publication, and lifetime.

LoadBalancerGroup<S>
|-- BackendView (Backends)
|   |-- ServiceDiscovery
|   `-- Arc<HealthRegistry>
|-- SelectorRebuildGate
|-- SelectorRebuildCancellation
`-- SelectorSlot<S> x N
    |-- config
    |-- SelectorRebuildState
    |   `-- pending SelectorRebuildRequest
    `-- ArcSwap<PublishedSelector<S>>
        |-- Arc<S>
        |-- readiness snapshot
        `-- SelectorReleaseGuard
            `-- SelectorReleaseSignal

Shared mode:
HealthCheckService ---> Arc<HealthRegistry> <--- other BackendViews

The main roles are:

• Backends, also named BackendView, owns discovered membership, enablement, health references, and the membership generation. Each published group selector owns the readiness snapshot for its generation, so older selectors keep serving their own snapshot.
• HealthRegistry reconciles the targets contributed by its views and owns one health state and probe target per backend equivalence key.
• HealthCheckService runs one active health-check loop for a shared registry. Views with private registries are checked by their load balancer.
• SelectorSlot<S> owns one selector configuration, its published selector, generation, pending work, timings, and counters.
• SelectorRebuildRequest contains the backend membership, its readiness snapshot, and generation to build.
• SelectorRebuildState tracks the active rebuild task and newest pending request.
• SelectorRebuildTaskGuard clears the running state and restores an in-flight request if the task exits unexpectedly.
• SelectorRebuildCancellation stops rebuild tasks when the group is dropped.
• PublishedSelector<S> pairs the selector exposed to requests with the readiness snapshot for its generation and its lifetime tracking.
• SelectorReleaseGuard and SelectorReleaseSignal notify the gate after a replaced selector and all its readers are gone.
• SelectorRebuildGate allows one build at a time and prevents another build while an old selector is still being destroyed.

Discovery and shared-health flow

1. LoadBalancerGroup<S> asks its BackendView to update.
2. The view’s ServiceDiscovery returns its current Backend membership and enablement.
3. BackendView publishes that membership and updates its contribution to HealthRegistry.
4. HealthRegistry reconciles the targets from all of its views.
5. In shared mode, HealthCheckService probes each registry target once.
6. The resulting health state is visible through every contributing view.
7. Each view applies its own membership and enablement. Each rebuilt group selector is published with the readiness snapshot for its generation.

Selector rebuild flow

1. Backends advances the membership generation and produces an indivisible membership and readiness update bundle.
2. LoadBalancerGroup<S> schedules each selector rebuild from that bundle.
3. SelectorRebuildState keeps one active rebuild and coalesces newer work into its pending request.
4. SelectorRebuildTaskGuard tracks the in-flight request while the task acquires SelectorRebuildGate.
5. The task builds the selector S from the request’s backend snapshot.
6. A new PublishedSelector<S> is stored in the slot’s ArcSwap, replacing the old published selector atomically.
7. The slot publishes its selector generation and can process its next pending request.

Request flow

1. LoadBalancerGroup<S>::select loads a PublishedSelector<S> from the chosen SelectorSlot<S>.
2. That published selector snapshot is held for the whole selection while it yields ordered backend candidates.
3. The published selector’s own readiness snapshot answers enablement and health for each candidate.
4. The first accepted backend is returned; otherwise selection returns None.
5. Replacing the selector does not affect this request’s iterator.
6. When the final reader releases the old PublishedSelector<S>, its SelectorReleaseGuard updates SelectorReleaseSignal.
7. SelectorRebuildGate observes that signal and permits the next build.

Cancellation flow

1. Dropping LoadBalancerGroup<S> triggers SelectorRebuildCancellation.
2. A task waiting for SelectorRebuildGate exits without removing the old selector’s SelectorReleaseSignal.
3. SelectorRebuildTaskGuard restores an in-flight SelectorRebuildRequest unless a newer request already replaced it.
4. A built but unpublished selector is destroyed on a blocking worker.
5. That destruction retains the gate permit, so another group cannot build at the same time.
6. The task guard clears the slot’s running state and notifies waiters.
7. After destruction completes, the gate permit is released.

Modules

discovery

Service discovery interface and implementations

health_check

Health Check interface and methods.

prelude

selection

Backend selection interfaces and algorithms

Structs

Backend

Backend represents a server to proxy or connect to.

Backends

A discovered backend membership with view-local enablement and shared active health state.

Extensions

A type map of protocol extensions.

HealthCheckService

Background service that runs one health-check loop for a shared HealthRegistry.

HealthRegistry

Active health state shared by one or more BackendViews.

LoadBalancer

A LoadBalancer instance contains the service discovery, health check and backend selection all together.

LoadBalancerGroup

A collection of eventually consistent load-balancing selectors that share one backend pool.

SelectorRebuildGate

A serial rebuild gate shared by one or more LoadBalancerGroups.

SelectorUpdateTimings

Timing information for one selector rebuild.

UpdateTimings

Timing information from the most recent LoadBalancer::update call.

Type Aliases

BackendView

Explicit name for a Backends instance used as one membership view.