Skip to main content

Crate dynamic_config_etcd

Crate dynamic_config_etcd 

Source
Expand description

Read dynamic-config configuration from an etcd v3 key/value store.

etcd speaks gRPC, so its client is async — which is why this implements the async AsyncRemoteSource trait rather than the blocking one.

use dynamic_config_etcd::Etcd;

DbConfig::set_remote_async(
    Etcd::new(["http://etcd.internal:2379"], "myapp/db.json").await?,
);

// Fetching is explicit; the load that follows touches no network.
DbConfig::refresh_remote_async().await?;

§What it reads

One key, whose value is a whole configuration document — the same bytes that would be in a config file. The format comes from the key’s extension, or from with_format.

§Several keys as one document

A deployment that splits its configuration across a range — myapp/db.json, myapp/server.json — can have one source read the lot, and Keys says which:

// Named keys: a list of layers, merged in the order given, later wins.
let etcd = Etcd::new(endpoints, Keys::several(["myapp/base.json", "myapp/local.json"])).await?;

// A prefix: disjoint sections, and an overlap between two of them is an error.
let etcd = Etcd::new(endpoints, Keys::prefix("myapp/"))
    .await?
    .with_format(dynamic_config::Format::Json);

Both are one round trip: a list is a transaction of range reads and a prefix is one range read, so either way the keys are read at a single etcd revision and a write landing mid-read cannot tear the document in half.

Three consequences, each of which belongs here rather than in an incident:

  • A prefix that matches more than 512 keys is refused. A prefix is caller input and the answer to it is server input; an empty prefix matches a whole cluster.
  • Provenance becomes store-grained. The merged document is one layer, so source_of answers “from etcd … keys a, b” and not which of them supplied a given value. describe names every key in the set, which is as close as one layer gets.
  • One unreadable key fails the whole fetch. A configuration quietly missing a section is worse than a refresh that failed and left the last document serving.

§The connection is made once, and lazily

Etcd::new builds the client and fetch reuses it — a source that reconnected on every read would turn a refresh loop into a connection storm.

The underlying client connects lazily, so new succeeding does not mean the endpoints are reachable: an unreachable etcd surfaces on the first fetch, not at construction. That is the client’s behaviour rather than a choice made here, and papering over it with an eager round trip would make every construction cost one.

§Timeouts

Etcd::with_timeout is the deadline for a single fetch attempt, excluding retries the underlying client performs — the sentence every store in this family answers to. Ten seconds by default.

etcd’s own ConnectOptions::with_timeout bounds connecting, which is a different thing and does not help a connection established minutes ago, so the deadline here wraps the request. Both can be set; they cover different halves. Neither applies to Etcd::watch, which is long-lived on purpose.

§Watching

etcd’s watch is a real push stream, so Etcd::watch is a future the caller spawns and cancels by dropping — no runtime is imposed and no flag is polled.

A prefix can be watched; a named list cannot. A watch on a set is only honest if the store says the set changed and the set can then be re-read as of one instant. A prefix answers both: one stream over the range says the set moved and carries the revision it moved at, and one range read at that revision is the whole subtree as one instant had it — so a delivered document is a state the cluster really was in, never one key’s new value merged with another’s old one. A named list answers neither: etcd establishes a watch on a key or a range, so a list is N independent streams, and none of them is about the set. That shape refuses at watch, before the first event; poll refresh_remote_async() on a timer instead — it is the same one round trip the fetch always was.

let task = tokio::spawn(async move {
    etcd.watch(move |document| sink(document)).await
});

// Dropping or aborting the task stops the watch.
task.abort();

§A watch that is failing says so

A watch is the half of a store dynamic-config cannot see: a delivery keeps RemoteStatus current, and a stream that broke delivers nothing and would otherwise report nothing — so dynamic_config_remote_up would describe the last delivery rather than the last attempt. reporting_to closes that: the sink the loop already holds is told about every attempt that came back with nothing, and a store that stopped answering an hour ago reads as down without anything having to call refresh_remote_async().

Structs§

Certificatetls
etcd’s TLS types, behind this crate’s tls feature.
Client
etcd’s own connection options, re-exported so authenticating needs no direct dependency on etcd-client. Asynchronous etcd client using v3 API.
ConnectOptions
etcd’s own connection options, re-exported so authenticating needs no direct dependency on etcd-client. Options for Connect operation.
Etcd
A key in etcd, as a configuration source.
Identitytls
etcd’s TLS types, behind this crate’s tls feature.
TlsConfig
A private certificate authority and a client certificate, as data.
TlsOptionstls
etcd’s TLS types, behind this crate’s tls feature.

Enums§

Keys
What a source reads: one key, several named keys, or a range.