Expand description
Read dynamic-config configuration from Consul’s key/value store.
Consul’s KV API is plain HTTP, so this implements the blocking
RemoteSource trait: nothing here needs an async runtime, and neither
does using it.
use dynamic_config_consul::Consul;
DbConfig::set_remote(
Consul::new("http://consul.internal:8500", "myapp/db.json")
.with_token(std::env::var("CONSUL_HTTP_TOKEN")?),
);
DbConfig::refresh_remote()?;§What it reads
GET {address}/v1/kv/{key}, and base64-decodes the single Value Consul
returns. The stored value is a whole configuration document — the same
bytes that would be in a config file — so the format comes from the key’s
extension, or from with_format.
That is the opposite of dynamic-config-vault, which wraps a secret’s
fields under a section key. The difference is not a whim: Vault stores a map
of named secrets, Consul stores an opaque blob, and each is easiest to use
as what it already is.
§Several keys as one document
A deployment that splits its configuration across a subtree — myapp/db,
myapp/server — can have one source read the lot, and Keys says which:
// Named keys: a list of layers, merged in the order given, later wins.
let consul = Consul::new(address, Keys::several(["myapp/base.json", "myapp/local.json"]));
// A prefix: disjoint sections, and an overlap between two of them is an error.
let consul = Consul::new(address, Keys::prefix("myapp/"))
.with_format(dynamic_config::Format::Json);The two forms cost different things, and the difference is the agent’s, not this crate’s:
- A prefix is one request —
?recurse, which Consul answers with the whole subtree at one index. So the set is consistent: a write landing mid-read cannot produce a document that never existed. - A named list is one request per key. Consul’s KV API reads one key or one subtree and has no batch read of a caller-chosen set, so a list is not read atomically. Its transaction endpoint could do it in one, at the price of a write-shaped request and a sixty-four operation ceiling; that trade is recorded rather than taken. Prefer a prefix where the keys are disjoint anyway.
Three consequences that belong here rather than in an incident:
- A prefix that matches more than 512 keys is refused. A prefix is caller input and the answer to it is server input.
- Provenance becomes store-grained. The merged document is one layer, so
source_ofanswers “from consul … keys a, b” rather than naming which key supplied a value.describenames the whole set, which is as close as one layer gets. - One unreadable key fails the whole fetch. A configuration quietly missing a section is worse than a refresh that failed and left the last document serving.
§Watching
Consul cannot push, but it can hold a request open until something changes —
a blocking query. Consul::watch is that loop, and it is genuinely
change-driven rather than a poll with extra steps: the agent answers the
moment the key moves.
It blocks, so it belongs on a thread, and a thread cannot be cancelled from
outside — hence the Watching token.
A prefix can be watched; a named list cannot. A watch on a set is only
honest if the store says the set changed and the set can then be read as
of one instant. A recursive blocking query answers both at once, and is the
only watch in this family that needs no re-read at all: the agent holds the
request open until the subtree’s index moves, and what it then sends back
is the subtree at that index. The document the callback receives is
folded from those exact bytes, so there is no window between noticing and
reading for a second write to land in. A named list has no such query —
Consul reads one key or one subtree, never a caller-chosen set — so it
refuses at watch, before the first change; poll
refresh_remote() on a timer instead.
let watch = RemoteWatch::new();
let watching = watch.watching();
std::thread::spawn(move || consul.watch(&watching, move |document| sink(document)));
// Dropping `watch` — or calling `watch.stop()` — ends the loop.§Every failure branch of the watch loop, and what it reports
A watch is the half of a store dynamic-config cannot see, and
reporting_to is what lets it speak: the sink the
loop already holds is told about every attempt that came back with
nothing. Which attempts those are is a table rather than prose, because
the question an operator asks is which silence is deliberate.
Three rules decide the column, and they are the same three in all seven store crates:
- A failure the loop survives by retrying reports. That is the case the whole feature exists for: the stream is down, the last delivery is old, and nothing else would ever say so out loud.
- A recovery that worked stays silent. Only a delivery or a fetch
clears the streak, so reporting a five-minute token turning over on a
healthy cluster would drive
remote_upto zero and leave it there. - A refusal that never asked the store reports nowhere. No format, a
key shape that cannot be watched, material that will not build a
client:
RemoteStatus::reachable()is whether the store answered the last time it was asked, and these never ask. They are returned to the caller, who is the one holding the mistake — and a status cannot correct them, since it carries a kind and a path and no message.
| Branch | Reports |
|---|---|
| the format is missing, the source cannot be watched, or the agent cannot be built | no — rule 3: nothing has been asked of the agent |
| the blocking query fails | yes, and the loop waits and retries |
| the subtree cannot be folded into one document | yes, and the watch ends — a deployment bug, not a blip |
| the watched key holds no value | yes, and the loop waits and retries |
| the index reset, the document is unchanged, or this is the priming query | no — the agent answered |
on_change refuses the document | no — the agent answered; apply counted the delivery, and what the document did next is ConfigStatus’s half |
The empty-key row is a difference between stores, deliberately left
standing: dynamic-config-etcd and dynamic-config-redis leave the
running snapshot alone and say nothing there, because only a delivery
clears a streak and a deleted key would park remote_up at zero. This
crate records it, on the argument that a fetch of the same key fails.
Both are written down at the branch, and neither moves in a patch release.
Re-exports§
Modules§
- auth
- Getting an ACL token, and getting another one when it stops working.
Structs§
- Consul
- A key in Consul’s KV store, as a configuration source.
- TlsConfig
- A private certificate authority and a client certificate, as data.
Enums§
- Keys
- What a source reads: one key, several named keys, or a subtree.