nopace
nopace is an asynchronous admission gate for rate-limited APIs. It decides
when a physical provider attempt may start, charges every budget that
attempt consumes, preserves capacity for important work, and learns stricter
pacing from provider responses.
It is intended for APIs such as GitHub, Meta, and Google Ads, where one request can consume several overlapping limits and where being a good citizen matters more than maximizing request volume.
Call -> acquire all claims atomically -> Permit -> provider attempt -> Feedback -> finish
| |
+-- Wait until safe ------+-- quota is already consumed
nopace does not own HTTP, serialization, retries, or circuit breaking. This
boundary is deliberate:
- your HTTP client still constructs and sends requests;
- your retry library still decides whether an error is retryable;
- every retry re-enters
nopace, so every physical attempt consumes quota; - a circuit breaker handles provider health, while
nopacehandles provider capacity.
Status
The current implementation includes the hardened core protocol and four store backends:
| Capability | Status |
|---|---|
| Fixed windows, including UTC day/month | Implemented |
| Exact sliding windows | Implemented |
| Token buckets | Implemented |
| Leased concurrency | Implemented |
| Atomic multi-claim admission | Implemented |
| Niceness, reserved headroom, and headroom snapshots | Implemented |
| Remaining/reset, not-before, and temporary-throttle feedback | Implemented |
| Closure, explicit permit, and typed request APIs | Implemented |
| Deterministic manual clock and contention tests | Implemented |
| Memory store | Implemented; one process only |
| PostgreSQL store | Implemented; transactional scope-row serialization |
| Redis store | Implemented; fenced atomic commits and server time |
| NATS store | Implemented; JetStream KV compare-and-swap, one key per partition |
Application-defined Store implementations |
Implemented; public extension contract |
Use the memory store when all callers sharing a budget live in one process.
Use PostgreSQL, Redis, NATS JetStream, or an application-owned registry for
multiple replicas. Every backend runs the same replay, completion, renewal, and
atomic admission contract, with one documented exception: NATS keys durable
state per partition, so a call whose claims span several partitions is refused
rather than admitted non-atomically. See
ScopeKey: the atomicity boundary.
Install
[]
= "0.1"
= { = "1", = ["macros", "rt-multi-thread"] }
The default memory feature re-exports MemoryStore. Core types can be used
without a backend:
= { = "0.1", = false }
Quick start
A token bucket allows an initial burst of two calls, then refills one unit every 100 ms:
use Duration;
use ;
# async
Construct one Pace at application startup and clone it into handlers. Cloning
a Pace or MemoryStore shares the same state; constructing another
MemoryStore::new() creates independent state.
The design in six types
A successful integration starts by modeling the provider's quota vocabulary rather than its HTTP endpoints.
ScopeKey: the atomicity boundary
A scope is configured once on Pace. All claims made through that gate are
evaluated and committed together.
Choose the broadest stable identity across which budgets can overlap. Examples:
- a GitHub credential fingerprint when primary and secondary limits share that credential;
- a Google Ads developer-token fingerprint when calls also consume customer and endpoint budgets;
- a Meta application ID when app, ad-account, and user pressure overlap.
On NATS, the partition is the atomicity boundary, not the scope. JetStream
KV gives compare-and-set on one key and no transaction across keys, and that
store keys durable state per (scope, partition) so one tenant's value cannot
grow with the tenant count. The last two examples above — several partitions in
one call — are therefore refused by NatsStore with
NatsError::CrossPartitionCall, loudly, rather than split into decisions that
would each let the other's traffic through. If a call genuinely spans
partitions, either give the group its own partition key or use PostgreSQL or
Redis, whose durable unit is the scope. Nothing changes for the common case of
one claim, or of several claims against one partition.
Do not create a scope per request. Do not put raw credentials in a scope or partition. Use a stable cryptographic fingerprint, ideally an HMAC with an application-owned key.
Policy: one provider budget family
A policy is a named, immutable collection of limits plus optional priority
headroom. For example, github-core might contain an hourly fixed window, a
secondary rolling-minute window, and a concurrency bound.
A claim must satisfy every limit in its policy. Keep policy and limit IDs stable across deployments; changing their meaning while retaining their names would reinterpret existing state in a future persistent backend.
PartitionKey: who shares a budget
The partition is the runtime identity charged by a claim:
- credential fingerprint;
- application ID;
- customer or ad-account ID;
- authenticated user ID;
- repository or endpoint group, if the provider defines such a budget.
Equal (policy, partition) pairs share state. Different partitions are
independent. Partition only where the provider partitions: inventing per-user
partitions for a credential-wide provider limit would multiply capacity and
over-admit.
Claim: one budget consumed by an attempt
A claim connects a stable label, policy, partition, and integer cost:
# use ;
#
One physical call can have several claims. Admission is all-or-none: if the customer budget is exhausted, the developer budget is not partially consumed. Costs are quota units, not necessarily request counts; a batch mutation can charge the number of operations it contains.
Claim labels are local to the call and let response feedback target the admitted state safely. Use unique labels within a call.
Identifiers are 1-128 bytes and may contain ASCII letters, digits, and
-._:/. This applies to call names, claim labels, policy IDs, limit IDs,
partitions, and scopes.
Call: one physical provider attempt
A call describes the attempt before it runs:
- a diagnostic name;
- all quota claims;
- niceness from
0(important) to19(background); - an optional
Instantdeadline.
Create a fresh Call for every physical retry. nopace does not retry closures
and does not decide whether a failed mutation is safe to repeat.
Permit: proof of admission
Quota is consumed when the permit is granted, before provider code starts.
finish applies response feedback and releases concurrency leases.
Dropping an unfinished permit is intentionally conservative:
- fixed/sliding/token quota remains consumed because the provider may have seen the request;
- concurrency remains occupied until its lease expires;
- there is no async refund from
Drop.
Always drive permits to finish, including provider failures, whenever the
process is still alive.
Limit algorithms
Choose the algorithm that matches the provider contract. A more expensive algorithm is not inherently safer if it models the wrong boundary.
| Limit | Use when | Behavior and tradeoff |
|---|---|---|
FixedWindow |
The provider resets at deterministic boundaries | Integer usage within epoch-aligned durations, UTC calendar days, or UTC calendar months. Cheap and compact. |
SlidingWindow |
The provider says “the preceding N seconds/minutes” | Stores exact admitted (timestamp, units) entries. Exact rolling semantics, but more state and work. |
TokenBucket |
The provider allows bursts with a steady refill rate | Integer capacity and refill. Smooth pacing without floating-point drift. |
Concurrency |
The provider caps simultaneous in-flight requests | A leased slot released by finish, recoverable after crashes through TTL. |
Several horizons belong in the same policy when they constrain the same partition:
# use Duration;
# use ;
#
Concurrency lease safety
A concurrency limit is a crash-recoverable bound, not a distributed mutex. If a process cannot renew and its request outlives the lease, another request may be admitted while the first is still running.
Current renewal is explicit:
# use Duration;
# use ;
# async
Set lease_ttl above the normal request duration and renew well before expiry.
Use the explicit permit API for long operations; Pace::run does not currently
renew automatically. For a hard side-effect mutex, use provider idempotency keys
or a database lock in addition to nopace.
Priority without a distributed scheduler
Niceness follows Unix intuition: 0 is most important, 19 is background.
Priority policies reserve capacity for more important work; reserve(threshold, basis_points) withholds that fraction from calls at that niceness and worse
rather than granting that fraction to them.
# use ;
#
This means:
- important calls can use the full provider capacity;
- normal calls stop after consuming their 80% usable share;
- background calls stop after consuming their 50% usable share;
- every admitted call still counts against the same hard provider limit;
- priority never increases provider capacity.
Reservations are integer basis points (10_000 = 100%) and must increase as
niceness increases. This preserves headroom but does not guarantee FIFO ordering
or prevent starvation caused by an unsuitable policy. Use
PriorityPolicy::reservations, reservation, or usable_capacity when tests
need to assert the configured policy directly. nopace is an admission gate,
not a distributed work scheduler.
Provider feedback
Static policy is the conservative starting point. Response feedback may make the next admission stricter based on what the provider actually reported.
FeedbackBuilder is created from a permit and accepts only claim/limit labels
that were admitted. This catches parser mistakes before unrelated state is
updated.
Remaining and reset
Use exact remaining quota and an absolute Unix-millisecond reset timestamp:
# use ;
#
Reported remaining is a ceiling on local admission. It cannot increase the configured static capacity.
Not before
Use not_before for Retry-After, explicit provider reset errors, or a
documented recovery time. Target one claim, or pass None to block every claim
carried by that permit.
Temporary throttle
Use temporary_throttle for provider pressure telemetry such as Meta usage
percentages. It overlays a temporary token bucket with a minimum interval,
burst, and expiry; once expired, the static policy remains.
Keep parsing provider-specific data outside nopace. The core deals only in
transport-neutral signals, which keeps GitHub headers, Meta JSON headers, and
Google Ads error payloads from becoming core dependencies.
Runtime headroom
Pace::headroom(call) returns the current headroom for every limit that would
gate that call, including learned provider constraints. The result is a snapshot,
not a reservation; admission can still change before the next acquire.
Three execution styles
| Need | API |
|---|---|
| Plain provider call; finish with no response feedback | run |
| Provider headers/body update pacing state | run_observed |
| Manual renewal, split parsing, or custom cleanup | acquire / Permit |
| Queue or scheduler owns waiting | try_acquire |
| Typed SDK wrapper | Request / execute |
| Dashboard or preflight budget snapshot | headroom |
Closure API: default for ordinary HTTP calls
run_observed keeps admission adjacent to the provider call and persists feedback
from both success and failure:
# use ;
# async
The closure is FnOnce; nopace never reruns it.
Explicit permit API: maximum control
Use acquire/finish when:
- provider output and feedback-store failure need separate handling;
- concurrency needs periodic renewal;
- response parsing spans several layers;
- cleanup must be coordinated explicitly.
RunError::Operation(error) displays the provider error, and
into_operation_error() extracts it when the retry policy wants the original
type. RunError::Feedback { outcome, source } is important: the provider
attempt already happened, and the complete success or failure outcome is
preserved even though feedback persistence failed. Do not blindly repeat a
mutation in this case.
One-shot admission: external queues and schedulers
Use try_acquire when another component owns waiting:
# use ;
# async
retry_at is an eligibility hint. Fixed windows, sliding windows, token
buckets, and learned reset times return Eligibility::At(_); saturated
concurrency returns Eligibility::NoLaterThan(_) because a permit may finish
before its lease TTL. acquire polls upper-bound hints using
PaceBuilder::uncertain_wait_interval.
Typed SDK wrappers can implement Request and use Pace::execute; the
request's call() remains visible and reviewable.
Put nopace inside the retry loop
The safe composition is:
retry policy
-> nopace acquire
-> one physical HTTP attempt
-> parse and persist feedback
-> classify result and maybe retry
Never acquire once around an entire retry loop. That would charge several
physical attempts as one and delay Retry-After feedback until retries are
already complete.
On a throttling failure:
- parse
Retry-After, remaining/reset, or provider pressure; - return
Observed::new(error).feedback(feedback); - let
run_observedfinish the permit and persist the signal; - only then let the retry policy decide whether another attempt is safe;
- the next attempt re-enters
nopaceand waits on the learned constraint.
Circuit breaking remains separate. A circuit breaker answers “is this dependency
healthy enough to try?”; nopace answers “would this attempt respect its
budgets?” A typical stack checks the circuit breaker before waiting for
admission, then records the provider outcome after the attempt.
Building a successful policy
Use this sequence for each provider integration.
1. Inventory authoritative limits
Record for every documented limit:
- identity it applies to: credential, app, customer, user, endpoint;
- capacity and quota unit;
- fixed, calendar, rolling, refill, or concurrency semantics;
- reset source and timestamp format;
- response/error fields that reveal stricter state;
- whether different endpoint costs exist.
Do not assume one request equals one unit.
2. Select one broad scope
Choose a scope containing every budget a physical call can touch. All processes sharing those budgets must use the same scope and the same store. A narrower scope duplicates capacity; an unnecessarily global scope adds contention.
3. Define stable policies and partitions
Name policies after provider budget families, not application functions. Derive partitions only from identities the provider actually isolates. Fingerprint credentials; never log or store raw secrets.
4. Use conservative static limits
Bootstrap from documented limits and leave safety margin where the provider is approximate or where other uncoordinated consumers exist. Feedback can tighten admission after a response, but the first call still needs a safe bound.
5. Attach every claim atomically
Describe every budget consumed by the physical attempt before it starts. If an
endpoint consumes developer-daily, customer-minute, endpoint-QPS, and
concurrency, include all four claims/policies in one Call.
6. Parse both success and failure responses
Rate information often arrives on successful responses; recovery times often
arrive on 429, 403, or provider-specific quota errors. Use run_observed
when preserving feedback on Observed<T> and Observed<E> is required.
7. Set deadlines and choose waiting ownership
Use Call::deadline(Instant) when a request becomes worthless after a point.
Use acquire when the task may wait in-process; use try_acquire when a
durable queue should own the delay. try_acquire exposes exact eligibility
times and upper-bound concurrency hints separately.
stores::ManualClock makes resets, rolling expiry, refill, feedback blocks, and
lease expiry deterministic. To test acquire/run waiting with it, pass the
same clock to both MemoryStore::with_clock(clock.clone()) and
PaceBuilder::clock(clock.clone()); otherwise only store decisions use manual
time. Tests should prove observable invariants:
- the N+1 call waits at the exact boundary;
- a multi-claim rejection changes no budget;
- important traffic can use reserved headroom;
- provider remaining
0blocks until reset; - retries consume one unit per physical attempt;
- dropped concurrency permits recover only after TTL;
- different provider partitions do not accidentally share state;
- identities that should share a budget really do.
8. Plan store failure semantics
With the memory store, admission is infallible but state disappears on process restart. Future distributed stores will return operational failures. Treat an admission-store failure as fail-closed unless violating provider limits is explicitly safer than delaying work. Treat feedback-store failure as an ambiguous completed provider operation, not as proof the provider call failed.
Bring your own store
Store is a public extension point. Applications can implement it for an
internal HTTP/gRPC registry, an existing database service, or another
coordination system:
# use ;
# ;
# ;
#
#
#
The store type must be cheap to clone. Wrap a shared client in Arc when that is
the natural ownership model; Arc<S> implements Store whenever S does:
# use Arc;
# use Duration;
# use ;
#
This accepts an Arc-wrapped custom store without an adapter crate.
Store semantics are the safety boundary
Implementing the Rust trait is the small part. The remote service must preserve the admission contract:
decideis linearizable over the state the command addresses: the scope, or the scope and partition for a store that keys state per partition.- Every claim in one request is checked and committed atomically. A store whose durable unit is narrower than the call must refuse the call, not split it.
- A grant consumes static quota, learned remaining, temporary throttles, and concurrency in one transaction.
- Repeating the same admission ID returns the same decision; it never charges twice.
finishis idempotent: duplicate delivery applies feedback once and releases concurrency once.renewextends only a live lease belonging to the permit and is safe to repeat.- Store/server time determines fixed boundaries, refill, feedback expiry, and lease expiry. Do not trust individual application clocks in a distributed deployment.
- A rejected multi-claim request changes no quota state.
- Unknown schema or algorithm versions fail closed rather than resetting quota.
- State and deduplication records live long enough for all active windows, throttles, leases, and transport retries.
AdmissionRequest contains resolved policies because the store is responsible
for executing their semantics atomically. A thin key/value registry is not
enough: a read followed by an application-side write races under contention.
Put the decision state machine in the authoritative registry, transaction, or
server-side atomic function.
For an API registry, use three idempotent commands:
POST /v1/admissions AdmissionRequest -> StoreDecision
POST /v1/leases/renew LeaseRenewal -> { "renewed": bool }
POST /v1/completions Completion -> { "finished": true }
Retry a command with the same payload and ID after timeouts. Do not manufacture a
new admission ID to resolve an ambiguous response: the first request may already
have committed. Return store outages as errors; Pace fails closed instead of
silently bypassing the gate.
The runnable custom-store example starts a real Axum server on an ephemeral
loopback port, builds a Reqwest-backed RegistryStore, and sends admission,
lease renewal, and completion as JSON over HTTP:
The server owns a map of ScopeState values and executes decide_at, renew_at,
and finish_at while holding one mutex across each complete state transition.
That makes the example a correct single-process registry rather than a
client-side read/modify/write simulation. Its integration tests also replay the
same admission ID over HTTP and prove that the result is idempotent.
The HTTP wire format is enabled by Serialize/Deserialize implementations on
the public store command and policy types. Call::deadline is intentionally
process-local and is not serialized; the client enforces it while waiting, and
the registry receives only the quota model needed for a decision.
The example server is not durable and loses state when it exits. A production registry must persist the same atomic transitions at its authoritative storage layer and define authentication, authorization, request-size limits, schema versioning, retention, and high availability. Before deployment, run contention, duplicate delivery, timeout-after-commit, multi-claim rollback, feedback, renewal, and TTL recovery tests against the real service.
Provider examples
The real-apis workspace example contains provider-specific adapters and
runnable scenarios:
# Anonymous or GITHUB_TOKEN + GITHUB_USER_ID
# META_ACCESS_TOKEN, META_APP_ID, META_AD_ACCOUNT_ID, META_USER_ID
# GOOGLE_ADS_URL, GOOGLE_ADS_BEARER_TOKEN, GOOGLE_ADS_DEVELOPER_TOKEN,
# GOOGLE_ADS_CUSTOMER_ID, GOOGLE_ADS_USER_ID
# Local demonstration of user/customer/credential partitioning
What they demonstrate:
| Provider | Static model | Learned signal |
|---|---|---|
| GitHub | credential hourly quota, secondary rolling points, concurrency, priority headroom | x-ratelimit-remaining, x-ratelimit-reset, Retry-After |
| Meta | app rolling budget, app/ad-account/user partitions, concurrency | x-app-usage, x-business-use-case-usage, recovery time |
| Google Ads | developer daily units, customer minute budget, optional endpoint QPS/concurrency | RESOURCE_EXHAUSTED, concurrent-request errors, Retry-After |
Provider limits change. Treat these as integration patterns, not a substitute for checking current provider documentation and limits assigned to your account.
Smaller examples:
Operational checklist
Before production:
- One long-lived
Pace/store is shared by all local handlers. - Every process consuming the same provider budget coordinates through one admission owner; memory state is not mistaken for distributed state.
- Scope is broad enough for every overlapping claim.
- Raw tokens and credentials never appear in scope, partition, logs, or metrics.
- Policy/limit IDs and partition derivation are stable.
- Static limits are conservative and use correct quota units.
- Every physical retry creates a fresh call and re-enters the gate.
- Success and throttling responses both emit feedback.
- Long operations use adequate concurrency TTL and explicit renewal.
-
RunError::Feedbackis handled as a completed provider attempt. - Deadlines prevent obsolete work from waiting indefinitely.
- Deterministic boundary and contention tests cover the actual provider model.
Crates and features
nopace: facade crate and the recommended application dependency.nopace-core: transport-independent model, algorithms, execution API, andStorecontract.nopace-memory: cloneable process-local store with system and manual clocks.nopace-postgres: transactional durable PostgreSQL store. It owns onetokio_postgres::Clientand serializes operations through it; services that need an existing pool should wrap their ownStore.nopace-redis: atomic Redis store with fenced operation commits.nopace-nats: durable NATS JetStream KV store. One compare-and-set key per(scope, partition), so a value's size follows one tenant's traffic rather than the tenant count, and unrelated tenants never share a revision. A delete or purge tombstone resets a partition. Because two partitions are two keys with no shared transaction, a call whose claims span several partitions is refused rather than split; seeDESIGN.mdsection 6.4.
Facade features:
| Feature | Default | Effect |
|---|---|---|
memory |
yes | Re-exports MemoryStore and clock types |
postgres |
no | Re-exports transactional PostgresStore |
redis |
no | Re-exports fenced atomic RedisStore |
nats |
no | Re-exports JetStream KV NatsStore |
See DESIGN.md for the distributed-store contracts, backend
atomicity strategy, and acceptance matrix.