Struct histlog::HistLog

pub struct HistLog { /* private fields */ }

Provides off-thread serialization of HdrHistogram interval logs to file.

Purpose

HdrHistogram is often used to measure latency. Generally, if something is important enough to measure latency, it’s unlikely you want to write to a file on the same thread.

One option would be to serialize to an in-memory buffer (e.g. Vec<u8>). However, this would still require allocating to the buffer, and would eventually require a lot of memory for a long-running process.

HistLog allows the hot thread to pass off it’s hdrhistogram::Histogram at regular intervals to a designated writer thread that can afford to dilly dally with IO. The interval log is written incrementally and can be inspected and analyzed while the program is still running.

HistLog relies completely on the rust port of HdrHistogram, both for the in-memory recording of values and serialization. What it does provide is off-thread writing with a clean interface and sane defaults that make it relatively easy to use.

Examples

A HistLog has a “series” name and a “tag.” The HdrHistogram interval log format provides for one tag per entry. The series name is used to name the file the interval log is written to:

use std::time::*;

let log_dir = "/tmp/path/to/logs";
let series = "server-latency";          // used to name the log file
let tag = "xeon-e7-8891-v2";            // recorded with each entry
let freq = Duration::from_secs(1);      // how often results sent to writer thread

// `HistLog::new` could fail creating file, `hdrhistogram::Histogram`
let mut server1 = histlog::HistLog::new(log_dir, series, tag, freq).unwrap();

// use `HistLog::clone_with_tag` to serialize a separate tag to same file.
let mut server2 = server1.clone_with_tag("xeon-e5-2670");

for i in 0..1000u64 { // dummy data
    server1.record(i).unwrap(); // call to `hdrhistogram::Histogram::record` could fail
    server2.record(i * 2).unwrap();
}

assert_eq!(server1.path(), server2.path()); // both being saved to same file, via same writer thread

HistLog’s api design is built for event loops. Each iteration of the loop, new values are recorded, and the current time is checked to see whether the current Histogram should be passed off to the writer thread:

use std::time::*;
#[cfg(feature = "minstant")]
use minstant::Instant;

let mut spintime = histlog::HistLog::new("/tmp/var/hist", "spintime", "main", Duration::from_secs(60)).unwrap();

let mut loop_time = Instant::now();
let mut prev: Instant;

loop {
    prev = loop_time;
    loop_time = Instant::now();
    spintime.record(histlog::nanos(loop_time - prev)).unwrap(); // nanos: Duration -> u64
    spintime.check_send(loop_time); // sends to writer thread if elapsed > freq,
    // or...
    spintime.check_try_send(loop_time).unwrap(); // non-blocking equivalent (can fail)

    // do important stuff ...

}

Logs

Logs are saved to <log dir>/<series name>.<datetime>.hdrhistogram-interval-log.v2.gz.

Format of log is like this:

#[StartTime: 1544631293.283 (seconds since epoch)]
#[BaseTime: 0.000 (seconds since epoch)]
Tag=xeon-e7-8891-v2,1544631293.283,0.003,999.000,HISTFAAAAC94Ae3GMRUAMAgD0bRI6FovNVcHmGREAgNR [...]
Tag=xeon-e5-2670,1544631293.283,0.003,999.000,HISTFAAAABx4AZNpmSzMwMDAxAABzFCaEUoz2X+AsQA/awK [...]
[...]

Only the histogram data is compressed (deflate), so a .gz extension is perhaps misleading.

Log file can be viewed/analyzed here (javascript, runs locally) or with the Java-based HistogramLogAnalyzer.

Full documentation of log serialization available from the hdrhistogram crate.

Features

• minstant: use minstant::Instant as a faster replacement for std::time::Instant
• smol_str: switch &'static str for smol_str::SmolStr for the SeriesName and Tag types, allowing dynamic string values to be provided instead of static strings.

Limitations

• HistLog::check_send and HistLog::check_try_send create a new hdrhistogram::Histogram and send the current/prev one to the writer thread each interval. Internally, an hdrhistogram::Histogram uses a Vec to store its counts, so there’s an allocation involved.
• Only u64 values can be recorded, currently.

Implementations

impl HistLog

pub fn new<P>( save_dir: P, series: SeriesName, tag: Tag, freq: Duration ) -> Result<Self, Error>

where P: AsRef<Path>,

Create a new HistLog.

If save_dir does not exist, will attempt to create it (which could fail). Creating a new log file could fail. Spawning the writer thread could fail.

Default significant figures of 3 is used.

pub fn new_with_sig_fig<P>( sig_fig: u8, save_dir: P, series: SeriesName, tag: Tag, freq: Duration ) -> Result<Self, Error>

where P: AsRef<Path>,

Create a new HistLog, specifying the number of significant digits

pub fn new_with_max<P>( max: u64, sig_fig: u8, save_dir: P, series: SeriesName, tag: Tag, freq: Duration ) -> Result<Self, Error>

where P: AsRef<Path>,

Create a new HistLog, specifying the max value and number of significant digits

pub fn new_from<P>( template_hist: &Histogram<C>, save_dir: P, series: SeriesName, tag: Tag, freq: Duration ) -> Result<Self, Error>

where P: AsRef<Path>,

Create a new HistLog by providing a template hdrhistogram::Histogram for it to use in recording the interval logs.

pub fn path(&self) -> &Path

Returns the path of the log file the HistLog is writing to.

pub fn clone_with_tag(&self, tag: Tag) -> Self

Record a new histogram with a tag that will serialize to the same interval log file as its parent. Each cloned HistLog’s entries will be written to their own lines in the log file, identifiable by tag.

Limitations

No effort is made to check whether tag is a duplicate of a previous tag, and using a duplicate may produce unexpected results.

pub fn record(&mut self, value: u64) -> Result<(), Error>

Record a single value to the histogram. This could fail if the value is outside of the highest range permitted. See the hdrhistogram docs for further deails. The hdrhistogram::Histogram used by HistLog is created with a significant figure of 3 (histlog::SIG_FIG const).

pub fn saturating_record(&mut self, value: u64)

From hdrhistogram docs:

Record value in the histogram, clamped to the range of the histogram.

This method cannot fail, as any values that are too small or too large to be tracked will automatically be clamed to be in range. Be aware that this will hide extreme outliers from the resulting histogram without warning. Since the values are clamped, the histogram will also not be resized to accomodate the value, even if auto-resize is enabled.

pub fn reset(&mut self)

Reset the state of the internal histogram and the last sent value.

One situation this might be used is if there was a pause in recording.

pub fn check_send(&mut self, loop_time: Instant) -> bool

Send the current histogram to the writer thread if the elapsed time since the last send is greater than the interval frequency.

If the channel is disconnected, this will fail silently, instead of panicking.

pub fn check_try_send(&mut self, loop_time: Instant) -> Result<bool, Error>

Non-blocking variant of HistLog::check_send, which will also return any errors, including a disconnected channel, encountered while trying to send to the writer thread.

Trait Implementations

impl Clone for HistLog

fn clone(&self) -> Self

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Drop for HistLog

fn drop(&mut self)

Checks if the current instance is the last remaining instance with a reference to the underlying writer thread, and, if so, sends a terminate signal to the writer thread and attempts to join it.

May Pause Up To 100ms

In the event the channel to the writer thread is full, will continue trying to send a terminate command (busy polling the channel) until DROP_DEADLINE has expired (currently 5ms), upon which it will abort.

If channel is disconnected, will simply abort without trying to join the writer thread.