Skip to main content

Crate nopace_nats

Crate nopace_nats 

Source
Expand description

Durable NATS JetStream KV store for nopace.

§What one key holds

One key holds the state of one partition inside one scope, and nothing else. That is the whole design, and it is worth being explicit about why, because the obvious alternative — one key per scope — is what this crate used to do and it took a service down for nine and a half hours.

A scope is the budget’s identity, so applications reasonably configure one constant scope and put the customer in the partition. Serializing a scope meant serializing every partition in it into a single KV value, so that value grew linearly with the tenant count. At about two thousand tenants it crossed the server’s max_payload, NATS refused the write, and because the write error was discarded the store retried the same oversized value for ever. Every admission blocked, every request failed closed, and no counter moved. Keying by partition makes the value’s size a function of one tenant’s traffic instead of the whole customer base, and it is also what stops unrelated tenants from contending on one revision.

§Delete is a reset

Deleting or purging a KV key does not remove it; it appends a tombstone, which Store::entry returns with an empty value and a real revision. The previous implementation fed that empty value to the deserializer and returned “invalid state” for ever after, so the documented operator mitigation — nats kv del — bricked the key instead of resetting it. The same went for writing an empty value by hand, which is an ordinary put and not a tombstone at all. Anything empty is now read as absent state, while its revision is still used for the next compare-and-set, so a reset is a reset and the write that follows it is still atomic.

Bytes that are present but do not decode stay an error: that is corruption or an incompatible build, not a reset, and quietly discarding live quota for it would be worse than refusing.

§What this backend cannot do

All claims of one call are admitted atomically. Two partitions are two keys, and two KV keys have no shared transaction, so a call whose claims span more than one partition is refused with NatsError::CrossPartitionCall rather than silently split into independent decisions that would each let the other’s traffic through. Model such a call as one partition — the group’s identity — or use a backend whose durable unit is the scope.

Structs§

NatsStore

Enums§

NatsError