[−][src]Crate async_logger
AsyncLoggerNB
is implementation of asynchronous logger/queue that allows writing arbitrary slices to a memory buffer,
and then send the buffer to a processing thread.
AsyncLoggerNB
uses pair of fixed size buffers;
while one buffer is being written by the multiple threads, the second is being proccessed by the
single "writer" thread. Writing to a buffers is lock-free operation.
Blocking appears only at the moment when buffers change their roles.
This makes AsyncLoggerNB
realy fast, and at the same time allows it be bounded.
It can be effectively used in mutlithreaded mutlicore environment with high level of concurrent writes
when you don't want to drop messages or run out of memory but still want to keep lock-free writes.
AsyncLoggerNB
can process serialized data (stream of bytes) or custom complex data structures, and also references to objects.
AsyncLoggerNB
can accept any "writer" as soon as it implements Writer
trait. This package includes
FileWriter
that writes data to a file.
Implementation of log facade based on this crate is available as separate crate async_logger_log.
Examples
use async_logger::FileWriter; use async_logger::AsyncLoggerNB; use std::{thread, sync::Arc}; let writer = FileWriter::new("/tmp", 10*1024*1024).expect("Failed to create file writer"); let logger = Arc::new(AsyncLoggerNB::new(Box::new(writer), 8192) .expect("Failed to create new async logger")); let write_line = "Hello, world!\n"; let logger_c = logger.clone(); let handle = thread::spawn(move || { logger_c.write_slice(write_line.as_bytes()).unwrap(); logger_c.write_slice(write_line.as_bytes()).unwrap(); logger_c.flush(); logger_c.write_slice(write_line.as_bytes()).unwrap(); }); handle.join().expect("Failed on thread join"); match Arc::try_unwrap(logger) { Ok(logger) => logger.terminate(), Err(_) => panic!("Failed to terminate logger because it is still in use"), };
When the size of data to be written is known in beforehand it may be more efficient to write data
directly to the underlying buffer. In this case AsyncLoggerNB::reserve_slice
can be used:
use async_logger::{FileWriter, AsyncLoggerNB, Writer}; // implement some custom writer along the way struct Stub {} impl Writer<u8> for Stub { fn process_slice(&mut self, slice: &[u8]) { for item in slice { println!("{}", item); } } fn flush(&mut self) {} } let logger = AsyncLoggerNB::new(Box::new(Stub {}), 8192) .expect("Failed to create new async logger"); // getting slice for writing let mut slice = logger.reserve_slice(10).unwrap(); assert_eq!(10, slice.len()); // write to the logger buffer directly for i in 0..10 { slice[i] = (i*i) as u8; } drop(slice); // release the buffer
Sometimes it is more efficient to write a pointer to some existing instance of struct instead
of copying the complete struct into buffer. This can be achieved by moving boxed reference to a struct to
AsyncLoggerNB::write_value
. See the documentation of the function
write_value for details and example.
Performance
Recommended buffer size is to let holding from tens to hundreds of messages. Choosing too small size leads to performance degradation. And choosing too big size doesn't increase performance significantly but leads to resource waste.
Performance tests
Tests show that this lock-free implementation is at least not slower than comparable implementation with mutex, and can be at least two times faster under highly competitive load.
Metrics
AsyncLoggerNB
collects total time spent by threads waiting for free buffer space in nanoseconds,
and total count of wait events.
Metrics collection is enabled at compile time with feature metrics
.
After enabling metrics AsyncLoggerNB::get_metrics
can be used to get the current metrics values.
Note, the metrics values can wrap around after significant amount of time of running without
interruption.
Notes
Attempt to get several instances of Slice
struct at the same time in the same thread can cause deadlock.
Structs
AsyncLoggerNB | Logger with non-blocking async processing. |
Error | Errors returned by the crate functions. |
FileWriter | Writer to a file. |
Metrics | Metrics values. |
Slice | Wrapper for writable slice of [T]. |
Enums
ErrorKind | Error kinds. |
Traits
Writer | Writer performs data processing of a fully filled buffer. |