Crate quicklog

Source
Expand description

An asynchronous single-threaded logger where formatting and I/O are deferred at callsite.

§Overview

Quicklog is provides a framework for logging where it allows for deferred deferred formatting and deferred I/O of logging, which should in turn provide more performant logging with low callsite latency.

§Deferred Formatting

§Why?

Formatting a struct into a String requires the overhead of serialization. Deferring the serialization of a struct can be avoided by cloning / copying the struct at a point in time, and saving that onto a queue.

Later at the flush site, this struct is serialized into a string when I/O is going to be performed.

§Deferred I/O

§Why?

Deferring the I/O of formatting would allow for low callsite latency and allow a user to implement their own flush site, possibly on a separate thread

§Usage

init!() macro needs to be called to initialize the logger before we can start logging, probably near the entry point of your application.

§Example Usage

fn main() {
    init!();

    // log some stuff
    info!("hello world! {}", "some argument");

    // flush on separate thread
    thread::spawn(|| {
        loop {
            flush!();
        }
    });
}

§Macros

§Shorthand Macros

Quicklog allows a number of macros with 5 different levels of verbosity. These wrap around log! with the corresponding levels.

Internally, these shorthands call try_log! with their respective levels.

§Setup Macros

Quicklog allows a user specified Clock or Flush to be implemented by the user. This can be passed in through these macros, as long as the underlying struct implements the correct traits

§Macro prefix for partial serialization

To speed things up, if you are logging a large struct, there could be some small things you might not want to log. This functionality can be done through implementing the Serialize trait, where you can implement how to copy which parts of the struct.

This could additionally be helpful if you already have the struct inside a buffer in byte form, as you could simply pass the buffer directly into the decode fn, eliminiating any need to copy.

struct SomeStruct {
    num: i64
}

impl Serialize for SomeStruct {
   fn encode(&self, write_buf: &'static mut [u8]) -> Store { /* some impl */ }
   fn buffer_size_required(&self) -> usize { /* some impl */ }
}

fn main() {
    let s = SomeStruct { num: 1_000_000 };
    info!("some struct: {}", ^s);
}

§Macro prefix for eager evaluation

There are two prefixes you can use for variables, % and ?. This works the same way as tracing, where % eagerly evaluates an object that implements Display and ? eagerly evaluates an object that implements Debug.

info!("eager display {}; eager debug {}", %impl_display, ?impl_debug);
// logically expands into:
// info!(
//      "eager display {}; eager debug {}",
//      format!("{}",   impl_display),
//      format!("{:?}", impl_debug)
// );

§Structured fields

Structured fields in log lines can be specified using field_name = field_value syntax. field_name can be a literal or a bunch of idents. This can also be used in combination with % and ? prefix on args to eagerly evaluate expressions into format strings.

info!("hello world {} {} {}", question.tricky = true, question.answer = ?value, question.val = &value);
// output: "hello world question.tricky=true question.answer=10 question.val=10"

§Environment variables

There are two environment variables you can set:

  1. QUICKLOG_MAX_LOGGER_CAPACITY
    • sets the size of the spsc ring buffer used for logging
  2. QUICKLOG_MAX_SERIALIZE_BUFFER_CAPACITY
    • sets the size of the byte buffer used for static serialization
    • this can be increased when you run into issues out of memory in debug when conducting load testing

§Components

§quicklog-clock

Clock is the trait for a clock that can be used with Quicklog. Clocks can be swapped out at runtime with a different implementation.

This swap should be done at the init stage of your application to ensure that timings are consistent.

§Example

struct SomeClock;

impl Clock for SomeClock { /* impl methods */ }

fn main() {
    init!();

    with_clock!(SomeClock::new());

    // logger now uses SomeClock for timestamping
    info!("Hello, world!");
    flush!();
}

§quicklog-flush

Flush is the trait that defines how the log messages would be flushed. These logs can be printed through using the pre-defined StdoutFlusher or saved to a file through the pre-defined FileFlusher to a specified location through the string passed in.

§Example

fn main() {
    init!();

    with_flush!(StdoutFlusher);

    // uses the StdoutFlusher passed in for flushing
    flush!();
}

Modules§

constants
constants.rs is generated from build.rs, should not be modified manually
level
contains logging levels and filters Defines the levels of verbosity available for logging, as well as LevelFilter.
macros
contains macros
serialize
contains trait for serialization and pre-generated impl for common types and buffer

Macros§

debug
Debug level log
error
Error level log
flush
Allows flushing onto an implementor of Flush, which can be modified with with_flush! macro and unwraps and ignores errors from try_flush
info
Info level log
init
Initializes Quicklog by calling Quicklog::init() Should only be called once in the application
log
Calls try_log and discards any errors
trace
Trace level log
try_flush
Allows flushing onto an implementor of Flush, which can be modified with with_flush! macro and returns RecvResult
try_log
Runs log and returns a Result, matches either a literal / or a literal with some arguments which are matched recursively.
warn
Warn level log
with_clock
Used to amend which Clock is currently attached to Quicklog An implementation can be passed in at runtime as long as it adheres to the Clock trait in quicklog-clock
with_flush
Used to amend which Flush is currently attached to Quicklog An implementation can be passed in at runtime as long as it adheres to the Flush trait in quicklog-flush
with_flush_into_file
Flushes log lines into the file path specified

Structs§

Quicklog
Quicklog implements the Log trait, to provide logging

Enums§

FlushError
Errors that can be presented when flushing

Constants§

MAX_LOGGER_CAPACITY
Sets max capacity of logging queue, can be set through env var QUICKLOG_MAX_LOGGER_CAPACITY.
MAX_SERIALIZE_BUFFER_CAPACITY
Sets max capacity of byte buffer used for serialization with ^ prefix in logging, can be set through QUICKLOG_MAX_SERIALIZE_BUFFER_CAPACITY.

Traits§

Log
Log is the base trait that Quicklog will implement. Flushing and formatting is deferred while logging.

Type Aliases§

Receiver
Consumer side of queue
RecvResult
Result from trying to pop from logging queue
SendResult
Result from pushing onto queue
Sender
Producer side of queue