Skip to main content

Crate dynamic_config_consul

Crate dynamic_config_consul 

Source
Expand description

Read dynamic-config configuration from Consul’s key/value store.

Consul’s KV API is plain HTTP, so this implements the blocking RemoteSource trait: nothing here needs an async runtime, and neither does using it.

use dynamic_config_consul::Consul;

DbConfig::set_remote(
    Consul::new("http://consul.internal:8500", "myapp/db.json")
        .with_token(std::env::var("CONSUL_HTTP_TOKEN")?),
);

DbConfig::refresh_remote()?;

§What it reads

GET {address}/v1/kv/{key}, and base64-decodes the single Value Consul returns. The stored value is a whole configuration document — the same bytes that would be in a config file — so the format comes from the key’s extension, or from with_format.

That is the opposite of dynamic-config-vault, which wraps a secret’s fields under a section key. The difference is not a whim: Vault stores a map of named secrets, Consul stores an opaque blob, and each is easiest to use as what it already is.

§Several keys as one document

A deployment that splits its configuration across a subtree — myapp/db, myapp/server — can have one source read the lot, and Keys says which:

// Named keys: a list of layers, merged in the order given, later wins.
let consul = Consul::new(address, Keys::several(["myapp/base.json", "myapp/local.json"]));

// A prefix: disjoint sections, and an overlap between two of them is an error.
let consul = Consul::new(address, Keys::prefix("myapp/"))
    .with_format(dynamic_config::Format::Json);

The two forms cost different things, and the difference is the agent’s, not this crate’s:

  • A prefix is one request?recurse, which Consul answers with the whole subtree at one index. So the set is consistent: a write landing mid-read cannot produce a document that never existed.
  • A named list is one request per key. Consul’s KV API reads one key or one subtree and has no batch read of a caller-chosen set, so a list is not read atomically. Its transaction endpoint could do it in one, at the price of a write-shaped request and a sixty-four operation ceiling; that trade is recorded rather than taken. Prefer a prefix where the keys are disjoint anyway.

Three consequences that belong here rather than in an incident:

  • A prefix that matches more than 512 keys is refused. A prefix is caller input and the answer to it is server input.
  • Provenance becomes store-grained. The merged document is one layer, so source_of answers “from consul … keys a, b” rather than naming which key supplied a value. describe names the whole set, which is as close as one layer gets.
  • One unreadable key fails the whole fetch. A configuration quietly missing a section is worse than a refresh that failed and left the last document serving.

§Watching

Consul cannot push, but it can hold a request open until something changes — a blocking query. Consul::watch is that loop, and it is genuinely change-driven rather than a poll with extra steps: the agent answers the moment the key moves.

It blocks, so it belongs on a thread, and a thread cannot be cancelled from outside — hence the Watching token.

A prefix can be watched; a named list cannot. A watch on a set is only honest if the store says the set changed and the set can then be read as of one instant. A recursive blocking query answers both at once, and is the only watch in this family that needs no re-read at all: the agent holds the request open until the subtree’s index moves, and what it then sends back is the subtree at that index. The document the callback receives is folded from those exact bytes, so there is no window between noticing and reading for a second write to land in. A named list has no such query — Consul reads one key or one subtree, never a caller-chosen set — so it refuses at watch, before the first change; poll refresh_remote() on a timer instead.

let watch = RemoteWatch::new();
let watching = watch.watching();

std::thread::spawn(move || consul.watch(&watching, move |document| sink(document)));

// Dropping `watch` — or calling `watch.stop()` — ends the loop.

§Every failure branch of the watch loop, and what it reports

A watch is the half of a store dynamic-config cannot see, and reporting_to is what lets it speak: the sink the loop already holds is told about every attempt that came back with nothing. Which attempts those are is a table rather than prose, because the question an operator asks is which silence is deliberate.

Three rules decide the column, and they are the same three in all seven store crates:

  1. A failure the loop survives by retrying reports. That is the case the whole feature exists for: the stream is down, the last delivery is old, and nothing else would ever say so out loud.
  2. A recovery that worked stays silent. Only a delivery or a fetch clears the streak, so reporting a five-minute token turning over on a healthy cluster would drive remote_up to zero and leave it there.
  3. A refusal that never asked the store reports nowhere. No format, a key shape that cannot be watched, material that will not build a client: RemoteStatus::reachable() is whether the store answered the last time it was asked, and these never ask. They are returned to the caller, who is the one holding the mistake — and a status cannot correct them, since it carries a kind and a path and no message.
BranchReports
the format is missing, the source cannot be watched, or the agent cannot be builtno — rule 3: nothing has been asked of the agent
the blocking query failsyes, and the loop waits and retries
the subtree cannot be folded into one documentyes, and the watch ends — a deployment bug, not a blip
the watched key holds no valueyes, and the loop waits and retries
the index reset, the document is unchanged, or this is the priming queryno — the agent answered
on_change refuses the documentno — the agent answered; apply counted the delivery, and what the document did next is ConfigStatus’s half

The empty-key row is a difference between stores, deliberately left standing: dynamic-config-etcd and dynamic-config-redis leave the running snapshot alone and say nothing there, because only a delivery clears a streak and a deleted key would park remote_up at zero. This crate records it, on the argument that a fetch of the same key fails. Both are written down at the branch, and neither moves in a patch release.

Re-exports§

pub use auth::Auth;
pub use auth::Bearer;

Modules§

auth
Getting an ACL token, and getting another one when it stops working.

Structs§

Consul
A key in Consul’s KV store, as a configuration source.
TlsConfig
A private certificate authority and a client certificate, as data.

Enums§

Keys
What a source reads: one key, several named keys, or a subtree.