Struct histlog::HistLog [−][src]
Provides off-thread serialization of HdrHistogram interval logs to file.
Purpose
HdrHistogram is often used to measure latency. Generally, if something is important enough to measure latency, it’s unlikely you want to write to a file on the same thread.
One option would be to serialize to an in-memory buffer (e.g. Vec<u8>
). However,
this would still require allocating to the buffer, and would eventually require a
lot of memory for a long-running process.
HistLog
allows the hot thread to pass off it’s hdrhistogram::Histogram
at regular intervals
to a designated writer thread that can afford to dilly dally with IO. The interval
log is written incrementally and can be inspected and analyzed while the program
is still running.
HistLog
relies completely on the rust port of HdrHistogram
, both for the in-memory
recording of values and serialization. What it does provide is off-thread writing with
a clean interface and sane defaults that make it relatively easy to use.
Examples
A HistLog
has a “series” name and a “tag.” The HdrHistogram interval log format provides
for one tag per entry. The series name is used to name the file the interval log is written to:
use std::time::*; let log_dir = "/tmp/path/to/logs"; let series = "server-latency"; // used to name the log file let tag = "xeon-e7-8891-v2"; // recorded with each entry let freq = Duration::from_secs(1); // how often results sent to writer thread // `HistLog::new` could fail creating file, `hdrhistogram::Histogram` let mut server1 = histlog::HistLog::new(log_dir, series, tag, freq).unwrap(); // use `HistLog::clone_with_tag` to serialize a separate tag to same file. let mut server2 = server1.clone_with_tag("xeon-e5-2670"); for i in 0..1000u64 { // dummy data server1.record(i).unwrap(); // call to `hdrhistogram::Histogram::record` could fail server2.record(i * 2).unwrap(); } assert_eq!(server1.path(), server2.path()); // both being saved to same file, via same writer thread
HistLog
’s api design is built for event loops. Each iteration of the loop, new values are
recorded, and the current time is checked to see whether the current Histogram
should be
passed off to the writer thread:
use std::time::*; let mut spintime = histlog::HistLog::new("/tmp/var/hist", "spintime", "main", Duration::from_secs(60)).unwrap(); let mut loop_time = Instant::now(); let mut prev: Instant; loop { prev = loop_time; loop_time = Instant::now(); spintime.record(histlog::nanos(loop_time - prev)).unwrap(); // nanos: Duration -> u64 spintime.check_send(loop_time); // sends to writer thread if elapsed > freq, // or... spintime.check_try_send(loop_time).unwrap(); // non-blocking equivalent (can fail) // do important stuff ... }
Logs
Logs are saved to <log dir>/<series name>.<datetime>.hdrhistogram-interval-log.v2.gz
.
Format of log is like this:
#[StartTime: 1544631293.283 (seconds since epoch)]
#[BaseTime: 0.000 (seconds since epoch)]
Tag=xeon-e7-8891-v2,1544631293.283,0.003,999.000,HISTFAAAAC94Ae3GMRUAMAgD0bRI6FovNVcHmGREAgNR [...]
Tag=xeon-e5-2670,1544631293.283,0.003,999.000,HISTFAAAABx4AZNpmSzMwMDAxAABzFCaEUoz2X+AsQA/awK [...]
[...]
Only the histogram data is compressed (deflate), so a .gz
extension is perhaps misleading.
Log file can be viewed/analyzed here (javascript, runs locally) or with the Java-based HistogramLogAnalyzer.
Full documentation of log
serialization available from the hdrhistogram
crate.
Limitations
- The series name and tags are currently limited to
&'static str
because the overhead of usingString
is prohibitive. This may change in future versions if a performant means of allowing dynamic tags presents itself that’s not inordinately complicated to use. HistLog::check_send
andHistLog::check_try_send
create a newhdrhistogram::Histogram
and send the current/prev one to the writer thread each interval. Internally, anhdrhistogram::Histogram
uses aVec
to store its counts, so there’s an allocation involved.- Only
u64
values can be recorded, currently.
Implementations
impl HistLog
[src]
pub fn new<P>(
save_dir: P,
series: &'static str,
tag: &'static str,
freq: Duration
) -> Result<Self, Error> where
P: AsRef<Path>,
[src]
save_dir: P,
series: &'static str,
tag: &'static str,
freq: Duration
) -> Result<Self, Error> where
P: AsRef<Path>,
Create a new HistLog
.
If save_dir
does not exist, will attempt to create it (which could
fail). Creating a new log file could fail. Spawning the writer thread could fail.
pub fn path(&self) -> &Path
[src]
Returns the path of the log file the HistLog
is writing to.
pub fn clone_with_tag(&self, tag: &'static str) -> Self
[src]
Record a new histogram with a tag
that will serialize to the
same interval log file as its parent. Each cloned HistLog
’s entries
will be written to their own lines in the log file, identifiable by tag.
Limitations
No effort is made to check whether tag
is a duplicate of a previous tag,
and using a duplicate may produce unexpected results.
pub fn record(&mut self, value: u64) -> Result<(), Error>
[src]
Record a single value to the histogram. This could fail if the value
is outside of the highest range permitted. See the
hdrhistogram
docs
for further deails. The hdrhistogram::Histogram
used by HistLog
is created with a significant figure of 3 (histlog::SIG_FIG
const).
pub fn reset(&mut self)
[src]
Reset the state of the internal histogram and the last sent value.
One situation this might be used is if there was a pause in recording.
pub fn check_send(&mut self, loop_time: Instant) -> bool
[src]
Send the current histogram to the writer thread if the elapsed time since the last send is greater than the interval frequency.
If the channel is disconnected, this will fail silently, instead of panicking.
pub fn check_try_send(&mut self, loop_time: Instant) -> Result<bool, Error>
[src]
Non-blocking variant of HistLog::check_send
, which will also return any errors,
including a disconnected channel, encountered while trying to send to the
writer thread.
Trait Implementations
impl Clone for HistLog
[src]
fn clone(&self) -> Self
[src]
pub fn clone_from(&mut self, source: &Self)
1.0.0[src]
impl Drop for HistLog
[src]
fn drop(&mut self)
[src]
Checks if the current instance is the last remaining instance with a reference to the underlying writer thread, and, if so, sends a terminate signal to the writer thread and attempts to join it.
May Pause Up To 5ms
In the event the channel to the writer thread is full, will continue trying
to send a terminate command (busy polling the channel) until DROP_DEADLINE
has expired (currently 5ms), upon which it will abort.
If channel is disconnected, will simply abort without trying to join the writer thread.
Auto Trait Implementations
impl !RefUnwindSafe for HistLog
impl Send for HistLog
impl Sync for HistLog
impl Unpin for HistLog
impl !UnwindSafe for HistLog
Blanket Implementations
impl<T> Any for T where
T: 'static + ?Sized,
[src]
T: 'static + ?Sized,
impl<T> Borrow<T> for T where
T: ?Sized,
[src]
T: ?Sized,
impl<T> BorrowMut<T> for T where
T: ?Sized,
[src]
T: ?Sized,
pub fn borrow_mut(&mut self) -> &mut T
[src]
impl<T> From<T> for T
[src]
impl<T, U> Into<U> for T where
U: From<T>,
[src]
U: From<T>,
impl<T> ToOwned for T where
T: Clone,
[src]
T: Clone,
type Owned = T
The resulting type after obtaining ownership.
pub fn to_owned(&self) -> T
[src]
pub fn clone_into(&self, target: &mut T)
[src]
impl<T, U> TryFrom<U> for T where
U: Into<T>,
[src]
U: Into<T>,
type Error = Infallible
The type returned in the event of a conversion error.
pub fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>
[src]
impl<T, U> TryInto<U> for T where
U: TryFrom<T>,
[src]
U: TryFrom<T>,