metrics/
lib.rs

1//! A lightweight metrics facade.
2//!
3//! The `metrics` crate provides a single metrics API that abstracts over the actual metrics implementation.  Libraries
4//! can use the metrics API provided by this crate, and the consumer of those libraries can choose the metrics
5//! implementation that is most suitable for its use case.
6//!
7//! # Overview
8//! `metrics` exposes two main concepts: emitting a metric, and recording it.
9//!
10//! ## Metric types, or kinds
11//! This crate supports three fundamental metric types, or kinds: counters, gauges, and histograms.
12//!
13//! ### Counters
14//! A counter is a cumulative metric that represents a monotonically increasing value which can only be increased or be
15//! reset to zero on restart. For example, you might use a counter to represent the number of operations performed, or
16//! the number of errors that have occurred.
17//!
18//! Counters are unsigned 64-bit integers.
19//!
20//! If you have a value that goes up and down over time, consider using a gauge.
21//!
22//! ### Gauges
23//! A gauge is a metric that can go up and down, arbitrarily, over time.
24//!
25//! Gauges are typically used for measured, external values, such as temperature, throughput, or things like current
26//! memory usage.  Even if the value is monotonically increasing, but there is no way to store the delta in order to
27//! properly figure out how much to increment by, then a gauge might be a suitable choice.
28//!
29//! Gauges support two modes: incremental updates, or absolute updates.  This allows callers to use them for external
30//! measurements -- where no delta can be computed -- as well as internal measurements.
31//!
32//! Gauges are floating-point 64-bit numbers.
33//!
34//! ### Histograms
35//! A histogram stores an arbitrary number of observations of a specific measurement and provides statistical analysis
36//! over the observed values.  Typically, measurements such as request latency are recorded with histograms: a specific
37//! action that is repeated over and over which can have a varying result each time.
38//!
39//! Histograms are used to explore the distribution of values, allowing a caller to understand the modalities of the
40//! distribution, such as whether or not all values are grouped close together, or spread evenly, or even whether or not
41//! there are multiple groupings or clusters.
42//!
43//! Colloquially, histograms are usually associated with percentiles, although by definition, they specifically deal
44//! with bucketed or binned values: how many values fell within 0-10, how many fell within 11-20, and so on and so
45//! forth.  Percentiles, commonly associated with "summaries", deal with understanding how much of a distribution falls
46//! below or at a particular percentage of that distribution: 50% of requests are faster than 500ms, 99% of requests are
47//! faster than 2450ms, and so on and so forth.
48//!
49//! While we use the term "histogram" in `metrics`, we enforce no particular usage of true histograms or summaries.  The
50//! choice of output is based entirely on the exporter being used to ship your metric data out of your application.  For
51//! example, if you're using [metrics-exporter-prometheus], Prometheus supports both histograms and summaries, and the
52//! exporter can be configured to output our "histogram" data as either.  Other exporters may choose to stick to using
53//! summaries, as is traditional, in order to generate percentile data.
54//!
55//! Histograms take floating-point 64-bit numbers.
56//!
57//! ## Emission
58//!
59//! Metrics are emitted by utilizing the emission methods.  There is a macro for registering and returning a handle for
60//! each fundamental metric type:
61//!
62//! - [`counter!`] returns the [`Counter`] handle then
63//!     - [`Counter::increment`] increments the counter.
64//!     - [`Counter::absolute`] sets the counter.
65//! - [`gauge!`] returns the [`Gauge`] handle then
66//!     - [`Gauge::increment`] increments the gauge.
67//!     - [`Gauge::decrement`] decrements the gauge.
68//!     - [`Gauge::set`] sets the gauge.
69//! - [`histogram!`] for histograms then
70//!     - [`Histogram::record`] records a data point.
71//!
72//! Additionally, metrics can be described -- setting either the unit of measure or long-form description -- by using
73//! the `describe_*` macros:
74//!
75//! - [`describe_counter!`] for counters
76//! - [`describe_gauge!`] for gauges
77//! - [`describe_histogram!`] for histograms
78//!
79//! In order to register or emit a metric, you need a way to record these events, which is where [`Recorder`] comes into
80//! play.
81//!
82//! ## Recording
83//!
84//! The [`Recorder`] trait defines the interface between the registration/emission macros, and exporters, which is how
85//! we refer to concrete implementations of [`Recorder`].  The trait defines what the exporters are doing -- recording
86//! -- but ultimately exporters are sending data from your application to somewhere else: whether it be a third-party
87//! service or logging via standard out.  It's "exporting" the metric data out of your application.
88//!
89//! Each metric type is usually reserved for a specific type of use case, whether it be tracking a single value or
90//! allowing the summation of multiple values, and the respective macros elaborate more on the usage and invariants
91//! provided by each.
92//!
93//! # Getting Started
94//!
95//! ## In libraries
96//!
97//! Libraries need only include the `metrics` crate to emit metrics.  When an executable installs a recorder, all
98//! included crates which emitting metrics will now emit their metrics to that record, which allows library authors to
99//! seamless emit their own metrics without knowing or caring which exporter implementation is chosen, or even if one is
100//! installed.
101//!
102//! In cases where no global recorder is installed, a "noop" recorder lives in its place, which has an incredibly
103//! low overhead: an atomic load and comparison.  Libraries can safely instrument their code without fear of ruining
104//! baseline performance.
105//!
106//! ### Examples
107//!
108//! ```rust
109//! use metrics::{counter, histogram};
110//!
111//! # use std::time::Instant;
112//! # pub fn run_query(_: &str) -> u64 { 42 }
113//! pub fn process(query: &str) -> u64 {
114//!     let start = Instant::now();
115//!     let row_count = run_query(query);
116//!     let delta = start.elapsed();
117//!
118//!     histogram!("process.query_time").record(delta);
119//!     counter!("process.query_row_count").increment(row_count);
120//!
121//!     row_count
122//! }
123//! # fn main() {}
124//! ```
125//!
126//! ## In executables
127//!
128//! Executables, which themselves can emit their own metrics, are intended to install a global recorder so that metrics
129//! can actually be recorded and exported somewhere.
130//!
131//! Initialization of the global recorder isn't required for macros to function, but any metrics emitted before a global
132//! recorder is installed will not be recorded, so initialization and installation of an exporter should happen as early
133//! as possible in the application lifecycle.
134//!
135//! ### Warning
136//!
137//! The metrics system may only be initialized once.
138//!
139//! For most use cases, you'll be using an off-the-shelf exporter implementation that hooks up to an existing metrics
140//! collection system, or interacts with the existing systems/processes that you use.
141//!
142//! Out of the box, some exporter implementations are available for you to use:
143//!
144//! * [metrics-exporter-tcp] - outputs metrics to clients over TCP
145//! * [metrics-exporter-prometheus] - serves a Prometheus scrape endpoint
146//!
147//! You can also implement your own recorder if a suitable one doesn't already exist.
148//!
149//! # Development
150//!
151//! The primary interface with `metrics` is through the [`Recorder`] trait, which is the connection between the
152//! user-facing emission macros -- `counter!`, and so on -- and the actual logic for handling those metrics and doing
153//! something with them, like logging them to the console or sending them to a remote metrics system.
154//!
155//! ## Keys
156//!
157//! All metrics are, in essence, the combination of a metric type and metric identifier, such as a histogram called
158//! "response_latency".  You could conceivably have multiple metrics with the same name, so long as they are of
159//! different types.
160//!
161//! As the types are enforced/limited by the [`Recorder`] trait itself, the remaining piece is the identifier, which we
162//! handle by using [`Key`]. Keys hold both the metric name, and potentially, labels related to the metric. The metric
163//! name and labels are always string values.
164//!
165//! Internally, `metrics` uses a clone-on-write "smart pointer" for these values to optimize cases where the values are
166//! static strings, which can provide significant performance benefits.  These smart pointers can also hold owned
167//! `String` values, though, so users can mix and match static strings and owned strings without issue.
168//!
169//! Two [`Key`] objects can be checked for equality and considered to point to the same metric if they are equal.
170//! Equality checks both the name of the key and the labels of a key.  Labels are _not_ sorted prior to checking for
171//! equality, but insertion order is maintained, so any [`Key`] constructed from the same set of labels in the same
172//! order should be equal.
173//!
174//! It is an implementation detail if a recorder wishes to do an deeper equality check that ignores the order of labels,
175//! but practically speaking, metric emission, and thus labels, should be fixed in ordering in nearly all cases, and so
176//! it typically is not a problem.
177//!
178//! ## Registration
179//!
180//! Recorders must handle the "registration" of a metric.
181//!
182//! In practice, registration solves two potential problems: providing metadata for a metric, and creating an entry for
183//! a metric even though it has not been emitted yet.
184//!
185//! Callers may wish to provide a human-readable description of what the metric is, or provide the units the metrics
186//! uses.  Additionally, users may wish to register their metrics so that they show up in the output of the installed
187//! exporter even if the metrics have yet to be emitted.  This allows callers to ensure the metrics output is stable, or
188//! allows them to expose all of the potential metrics a system has to offer, again, even if they have not all yet been
189//! emitted.
190//!
191//! As you can see from the trait, the registration methods treats the metadata as optional, and the macros allow users
192//! to mix and match whichever fields they want to provide.
193//!
194//! When a metric is registered, the expectation is that it will show up in output with a default value, so, for
195//! example, a counter should be initialized to zero, a histogram would have no values, and so on.
196//!
197//! ## Metadata
198//!
199//! When registering a metric, metadata can be provided to further describe the metric, in particular about where in the
200//! system it originates from and how verbose it is. This metadata emulates much of the same metadata as `tracing`, as
201//! it is intended to be used in a similar way: to provide the ability to filter metrics in a more granular way.
202//!
203//! Metadata provides three main pieces of information: the verbosity of the metric (level), the part of the system it
204//! originates from (target), and the Rust module it originates from (module path).
205//!
206//! For example, an application may wish to collect high-cardinality metrics, such as telemetry about a feature,
207//! including the customers using it. Tracking customer usage could mean having a tag with many possible values, and
208//! submitting these metrics to the configured downstream system could be costly or computationally expensive.
209//!
210//! By setting these metrics to a verbosity level of DEBUG, these metrics could potentially be filtered out at the
211//! recorder level, without having to change the application code or manually decide, at the callsite, whether or not to
212//! emit the metric.
213//!
214//! Metadata is exporter-specific, and may be ignored entirely. See the documentation of the specific exporter being
215//! used for more information on how metadata is utilized, if at all.
216//!
217//! ## Emission
218//!
219//! Likewise, recorders must handle the emission of metrics as well.
220//!
221//! Comparatively speaking, emission is not too different from registration: you have access to the same [`Key`] as well
222//! as the value being emitted.
223//!
224//! For recorders which temporarily buffer or hold on to values before exporting, a typical approach would be to utilize
225//! atomic variables for the storage.  For counters and gauges, this can be done simply by using types like
226//! [`AtomicU64`](std::sync::atomic::AtomicU64).  For histograms, this can be slightly tricky as you must hold on to all
227//! of the distinct values.  In our helper crate, [`metrics-util`][metrics-util], we've provided a type called
228//! [`AtomicBucket`][AtomicBucket].  For exporters that will want to get all of the current values in a batch, while
229//! clearing the bucket so that values aren't processed again, [AtomicBucket] provides a simple interface to do so, as
230//! well as optimized performance on both the insertion and read side.
231//!
232//! Combined together, exporter authors can use [`Handle`][Handle], also from the `metrics-util` crate, which provides a
233//! consolidated type for holding metric data.  These types, and many more from the `metrics-util` crate, form the basis
234//! of typical exporter behavior and have been exposed to help you quickly build a new exporter.
235//!
236//! ## Installing recorders
237//!
238//! Recorders, also referred to as exporters, must be "installed" such that the emission macros can access them. As
239//! users of `metrics`, you'll typically see exporters provide methods to install themselves that hide the nitty gritty
240//! details.  These methods will usually be aptly named, such as `install`.
241//!
242//! However, at a low level, this can happen in one of two ways: installing a recorder globally, or temporarily using it
243//! locally.
244//!
245//! ### Global recorder
246//!
247//! The global recorder is the recorder that the macros use by default. It is stored in a static variable accessible by
248//! all portions of the compiled application, including dependencies. This is what allows us to provide the same
249//! "initialize once, benefit everywhere" behavior that users are familiar with from other telemetry crates like
250//! `tracing` and `log`.
251//!
252//! Only one global recorder can be installed in the lifetime of the process. If a global recorder has already been
253//! installed, it cannot be replaced: this is due to the fact that once installed, the recorder is "leaked" so that a
254//! static reference can be obtained to it and used by subsequent calls to the emission macros, and any downstream
255//! crates.
256//!
257//! ### Local recorder
258//!
259//! In many scenarios, such as in unit tests, you may wish to temporarily set a recorder to influence all calls to the
260//! emission macros within a specific section of code, without influencing other areas of the code, or being limited by
261//! the constraints of only one global recorder being allowed.
262//!
263//! [`with_local_recorder`] allows you to do this by changing the recorder used by the emission macros for the duration
264//! of a given closure. While in that closure, the given recorder will act as if it was the global recorder for the
265//! current thread. Once the closure returns, the true global recorder takes priority again for the current thread.
266//!
267//! [metrics-exporter-tcp]: https://docs.rs/metrics-exporter-tcp
268//! [metrics-exporter-prometheus]: https://docs.rs/metrics-exporter-prometheus
269//! [metrics-util]: https://docs.rs/metrics-util
270//! [AtomicBucket]: https://docs.rs/metrics-util/0.5.0/metrics_util/struct.AtomicBucket.html
271//! [Handle]: https://docs.rs/metrics-util/0.5.0/metrics_util/enum.Handle.html
272#![deny(missing_docs)]
273#![cfg_attr(docsrs, feature(doc_cfg), deny(rustdoc::broken_intra_doc_links))]
274
275pub mod atomics;
276
277mod common;
278mod macros;
279pub use self::common::*;
280
281mod cow;
282
283mod handles;
284pub use self::handles::*;
285
286mod key;
287pub use self::key::*;
288
289mod label;
290pub use self::label::*;
291
292mod metadata;
293pub use self::metadata::*;
294
295mod recorder;
296pub use self::recorder::*;