Crate quicklog

source ·
Expand description

An asynchronous single-threaded logger where formatting and I/O are deferred at callsite.

Overview

Quicklog is provides a framework for logging where it allows for deferred deferred formatting and deferred I/O of logging, which should in turn provide more performant logging with low callsite latency.

Deferred Formatting

Why?

Formatting a struct into a String requires the overhead of serialization. Deferring the serialization of a struct can be avoided by cloning / copying the struct at a point in time, and saving that onto a queue.

Later at the flush site, this struct is serialized into a string when I/O is going to be performed.

Deferred I/O

Why?

Deferring the I/O of formatting would allow for low callsite latency and allow a user to implement their own flush site, possibly on a separate thread

Usage

init!() macro needs to be called to initialize the logger before we can start logging, probably near the entry point of your application.

Example Usage

fn main() {
    init!();

    // log some stuff
    info!("hello world! {}", "some argument");

    // flush on separate thread
    thread::spawn(|| {
        loop {
            flush!();
        }
    });
}

Macros

Shorthand Macros

Quicklog allows a number of macros with 5 different levels of verbosity. These wrap around log! with the corresponding levels.

Internally, these shorthands call try_log! with their respective levels.

Setup Macros

Quicklog allows a user specified Clock or Flush to be implemented by the user. This can be passed in through these macros, as long as the underlying struct implements the correct traits

Macro prefix for partial serialization

To speed things up, if you are logging a large struct, there could be some small things you might not want to log. This functionality can be done through implementing the Serialize trait, where you can implement how to copy which parts of the struct.

This could additionally be helpful if you already have the struct inside a buffer in byte form, as you could simply pass the buffer directly into the decode fn, eliminiating any need to copy.

ⓘ
struct SomeStruct {
    num: i64
}

impl Serialize for SomeStruct {
   fn encode(&self, write_buf: &'static mut [u8]) -> Store { /* some impl */ }
   fn buffer_size_required(&self) -> usize { /* some impl */ }
}

fn main() {
    let s = SomeStruct { num: 1_000_000 };
    info!("some struct: {}", ^s);
}

Macro prefix for eager evaluation

There are two prefixes you can use for variables, % and ?. This works the same way as tracing, where % eagerly evaluates an object that implements Display and ? eagerly evaluates an object that implements Debug.

info!("eager display {}; eager debug {}", %impl_display, ?impl_debug);
// logically expands into:
// info!(
//      "eager display {}; eager debug {}",
//      format!("{}",   impl_display),
//      format!("{:?}", impl_debug)
// );

Structured fields

Structured fields in log lines can be specified using field_name = field_value syntax. field_name can be a literal or a bunch of idents. This can also be used in combination with % and ? prefix on args to eagerly evaluate expressions into format strings.

info!("hello world {} {} {}", question.tricky = true, question.answer = ?value, question.val = &value);
// output: "hello world question.tricky=true question.answer=10 question.val=10"

Environment variables

There are two environment variables you can set:

  1. QUICKLOG_MAX_LOGGER_CAPACITY
    • sets the size of the spsc ring buffer used for logging
  2. QUICKLOG_MAX_SERIALIZE_BUFFER_CAPACITY
    • sets the size of the byte buffer used for static serialization
    • this can be increased when you run into issues out of memory in debug when conducting load testing

Components

quicklog-clock

Clock is the trait for a clock that can be used with Quicklog. Clocks can be swapped out at runtime with a different implementation.

This swap should be done at the init stage of your application to ensure that timings are consistent.

Example

ⓘ
struct SomeClock;

impl Clock for SomeClock { /* impl methods */ }

fn main() {
    init!();

    with_clock!(SomeClock::new());

    // logger now uses SomeClock for timestamping
    info!("Hello, world!");
    flush!();
}

quicklog-flush

Flush is the trait that defines how the log messages would be flushed. These logs can be printed through using the pre-defined StdoutFlusher or saved to a file through the pre-defined FileFlusher to a specified location through the string passed in.

Example

fn main() {
    init!();

    with_flush!(StdoutFlusher);

    // uses the StdoutFlusher passed in for flushing
    flush!();
}

Modules

  • constants
    constants.rs is generated from build.rs, should not be modified manually
  • level
    contains logging levels and filters Defines the levels of verbosity available for logging, as well as LevelFilter.
  • macros
    contains macros
  • serialize
    contains trait for serialization and pre-generated impl for common types and buffer

Macros

  • debug
    Debug level log
  • error
    Error level log
  • flush
    Allows flushing onto an implementor of Flush, which can be modified with with_flush! macro and unwraps and ignores errors from try_flush
  • info
    Info level log
  • init
    Initializes Quicklog by calling Quicklog::init() Should only be called once in the application
  • log
    Calls try_log and discards any errors
  • trace
    Trace level log
  • try_flush
    Allows flushing onto an implementor of Flush, which can be modified with with_flush! macro and returns RecvResult
  • try_log
    Runs log and returns a Result, matches either a literal / or a literal with some arguments which are matched recursively.
  • warn
    Warn level log
  • with_clock
    Used to amend which Clock is currently attached to Quicklog An implementation can be passed in at runtime as long as it adheres to the Clock trait in quicklog-clock
  • with_flush
    Used to amend which Flush is currently attached to Quicklog An implementation can be passed in at runtime as long as it adheres to the Flush trait in quicklog-flush
  • with_flush_into_file
    Flushes log lines into the file path specified

Structs

  • Quicklog
    Quicklog implements the Log trait, to provide logging

Enums

  • FlushError
    Errors that can be presented when flushing

Constants

  • MAX_LOGGER_CAPACITY
    Sets max capacity of logging queue, can be set through env var QUICKLOG_MAX_LOGGER_CAPACITY.
  • MAX_SERIALIZE_BUFFER_CAPACITY
    Sets max capacity of byte buffer used for serialization with ^ prefix in logging, can be set through QUICKLOG_MAX_SERIALIZE_BUFFER_CAPACITY.

Traits

  • Log
    Log is the base trait that Quicklog will implement. Flushing and formatting is deferred while logging.

Type Definitions