Crate prometheus_metric_storage

Derive macro to instantiate and register prometheus metrics without having to write tons of boilerplate code.

Motivation

When instrumenting code with prometheus metrics, one is required to write quite a bit of boilerplate code.

Creating metrics, setting up their options, registering them, having to store metrics in some struct and pass this struct around, all of this is cumbersome to say the least.

The situation is partially alleviated by using the static metrics mechanism, that is, metrics defined within a lazy_static!. This approach is limited, though. It relies on the default registry which can’t be configured, it requires having a global state, and it’s not suitable for libraries.

All in all, one usually ends up with something like this:

struct Metrics {
    inflight: prometheus::IntGauge,
    requests_duration_seconds: prometheus::Histogram,
};

impl Metrics {
    fn new(registry: &prometheus::Registry) -> prometheus::Result<Self> {
        let opts = prometheus::Opts::new(
            "inflight",
            "Number of requests that are currently inflight."
        );
        let inflight = prometheus::IntGauge::with_opts(opts)?;

        let opts = prometheus::HistogramOpts::new(
            "requests_duration_seconds",
            "Processing time of each request in seconds."
        );
        let requests_duration_seconds = prometheus::Histogram::with_opts(opts)?;

        Ok(Self {
            inflight,
            requests_duration_seconds,
        })
    }
}

This crate provides a derive macro that can automatically generate the new function for the above struct.

Quickstart

Define a struct that contains all metrics for a component and derive the MetricStorage trait:

#[derive(MetricStorage)]
struct Metrics {
    /// Number of requests that are currently inflight.
    inflight: prometheus::IntGauge,

    /// Processing time of each request in seconds.
    requests_duration_seconds: prometheus::Histogram,
}

Now you can instantiate this struct and register all metrics without having to write lots of boilerplate code:

let registry = prometheus::Registry::default();
let metrics = Metrics::new(&registry).unwrap();
metrics.inflight.inc();
metrics.requests_duration_seconds.observe(0.25);

Field names become metric names, and first line of each of the field’s documentation becomes metric’s help message. Additional configuration can be done via the #[metric(...)] attribute.

So, the code above will report the following metrics:

# HELP inflight Number of requests that are currently inflight.
# TYPE inflight gauge
inflight 1
# HELP requests_duration_seconds Processing time of each request in seconds.
# TYPE requests_duration_seconds histogram
requests_duration_seconds_bucket{le="0.005"} 0
...
requests_duration_seconds_sum 0.25
requests_duration_seconds_count 1

Generated code API

The derive macro will automatically generate implementation for the MetricStorage trait. On top of it, it will generate three more methods:

• fn new(registry: &Registry, …) -> Result<Self>:

  Creates a new instance of a metric storage and registers all of its metrics in the given registry via MetricStorage::register.

  This method accepts a reference to a registry, and const labels, if storage defines any (see section on configuring metrics, and also the const_labels field of the prometheus::Opts struct).

  Const label parameters should implement Into<String>, they are listed in the same order as they appear in the #[metric(labels(...))] attribute.

• fn new_unregistered(…) -> Result<Self>:

  Same as new, but doesn’t add metrics to any registry. You can use MetricStorage::register to register metrics later.

• fn instance(registry: &StorageRegistry, …) -> Result<&Self>:

  Looks up storage with the given const label values in a StorageRegistry, creates one if it’s not found.

  See StorageRegistry::get_or_create_storage for more info.

Configuring metrics

Additional configuration can be done via the #[metric(...)] attribute.

On the struct level the available keys are the following:

• subsystem — a string that will be prepended to each metrics’ name.

  For example, consider the following storage:

  #[derive(MetricStorage)]
  #[metric(subsystem = "transport")]
  struct Metrics {
      /// Processing time of each request in seconds.
      requests_duration_seconds: prometheus::Histogram,
  }

  Here, the metric will be named transport_requests_duration_seconds.

  See the subsystem field of the prometheus::Opts struct for more info on components that constitute a metric name.

• labels — a list of const labels that will be added to each metric.

  These labels should be provided during the storage initialization. They allow registering multiple metrics with the same name and different const label values. Or, in case of this crate, creating and registering multiple instances of the same metric storage.

  Note, however, that trying to register the same storage with the same const label values twice will still lead to an “already registered” error. To bypass this, use metric storage registry.

  Example:

  #[derive(MetricStorage)]
  #[metric(labels("url"))]
  struct Metrics {
     ...
  }
  
  let google_metrics = Metrics::new(&registry, "https://google.com/").unwrap();
  
  // This will not return an error because we're using a different label value.
  let duckduckgo_metrics = Metrics::new(&registry, "https://duckduckgo.com/").unwrap();
  
  // This will return an error because metric storage with the same label value
  // is already registered:
  // let google_metrics_2 = Metrics::new(&registry, "https://google.com/").unwrap();

  See the const_labels field of the prometheus::Opts struct for more info on different label settings.

On the field level, the following options are available:

• name — a string that overrides metric name derived from the field name.

  This is useful for tuple structs:

  #[derive(MetricStorage)]
  struct Metrics (
      #[metric(name = "requests", help = "Number of successful requests.")]
      prometheus::IntCounter
  );

  Note that this setting does not override subsystem configuration. That is, subsystem will still be prepended to metric’s name.

• help — a string that overrides help message derived from documentation.

• labels — a list of strings that will be used as labels for multidimensional (Vec) metrics. Order of labels will be preserved, so you can rely on it in functions such as MetricVec::with_label_values.

  Example:

  #[metric(labels("url", "status"))]
  requests_finished: prometheus::IntCounterVec,

• buckets — a list of floating point numbers used as histogram bucket bounds. Numbers should be listed in ascending order.

  Example:

  #[metric(buckets(0.1, 0.2, 0.5, 1, 2, 4, 8))]
  requests_duration_seconds: prometheus::Histogram,

Supporting custom collectors

If your project uses custom collectors, metric storage will not be able to instantiate them by default. You’ll have to implement MetricInit and possibly HistMetricInit for each of the collector you wish to use.

Metric storage registry

When registering a metric storage, there’s a requirement that a single metric should not be registered twice within the same registry. In practice, this means that, once a storage has been created and registered, it should not be created again:

fn do_stuff() {
    ...

    let metrics = Metrics::new(prometheus::default_registry()).unwrap();
    metrics.pieces_of_stuff_processed.inc();
}

...

// The first call will work just fine.
do_stuff();

// The second call will panic, though, because the metrics
// were already registered.
do_stuff();

There are two approaches to solve this issue.

The first one is to create a static variable within the do_stuff’s body. As was pointed out in the section about motivation, the code will have to rely on the default registry, so this is not suitable for libraries.

The second one is to have do_stuff accept a reference to Metrics. This solution complicates component’s public API, exposes implementation details of metric collection.

To give a better way of dealing with the situation, this crate provides StorageRegistry: a wrapper around the default registry that keeps track of all created storages, and makes sure that a single storage is only registered once:

fn do_stuff(registry: &StorageRegistry) {
    ...

    let metrics = Metrics::instance(registry).unwrap();
    metrics.pieces_of_stuff_processed.inc();
}

...

let registry = StorageRegistry::default();

// The first call will work just fine.
do_stuff(&registry);

// The second call will also work, because `StorageRegistry`
// makes sure not to register storages twice.
do_stuff(&registry);

Structs

StorageRegistry

Wrapper for prometheus’ Registry that keeps track of registered storages, and helps to avoid “already registered” errors without having to use lazy statics.

Traits

HistMetricInit

This trait is used to initialize metrics that accept buckets.

MetricInit

This trait is used to initialize metrics.

MetricStorage

Common interface for metric storages.

Functions

default_storage_registry

Get the default storage registry that uses prometheus::default_registry.

Derive Macros

MetricStorage

Generates implementation for MetricStorage and three additional methods: new, new_unregistered, instance.