log-nonblock

A high-performance Rust logging library that implements truly non-blocking writes to STDOUT/STDERR.

Features

Non-blocking writes: Log messages are queued and written asynchronously
Timestamps: ISO 8601 timestamps with UTC or custom timezone offset
Colors: Colored log levels for terminal output
JSON output: Structured JSON logging for log aggregators (Datadog, New Relic, etc.)
Thread names: Optional thread identification in log messages

Usage

[dependencies]
log_nonblock = "0.1"
log = "0.4"

use log_nonblock::NonBlockingLoggerBuilder;

fn main() {
    let logger = NonBlockingLoggerBuilder::new()
        .init()
        .unwrap();

    log::info!("Server started");
    log::warn!("Low memory");
    log::error!("Connection failed");

    log::logger().flush();
}

Output:

2024-01-26T12:34:56.789Z INFO  [my_app] Server started
2024-01-26T12:34:56.790Z WARN  [my_app] Low memory
2024-01-26T12:34:56.790Z ERROR [my_app] Connection failed

JSON Logging

Enable the json feature for structured JSON output:

[dependencies]
log_nonblock = { version = "0.1", features = ["json"] }

use log_nonblock::NonBlockingLoggerBuilder;

let logger = NonBlockingLoggerBuilder::new()
    .with_json()
    .init()
    .unwrap();

log::info!("Server started");
log::error!("Connection failed");

Output:

{"timestamp":"2024-01-26T12:34:56.789Z","level":"info","target":"my_app","message":"Server started"}
{"timestamp":"2024-01-26T12:34:56.790Z","level":"error","target":"my_app","message":"Connection failed"}

Motivation

Problem #1: STDOUT/STDERR Writes Are Synchronous Blocking Operations

Writing to STDOUT or STDERR is a slow I/O operation, and by default in Rust (and most languages), these are synchronous blocking calls:

println!("Log message");  // Your thread STOPS here until the write completes
log::info!("Log message"); // Same - blocks until written

When you write to STDOUT/STDERR, your application thread stops and waits until the operating system completes the write operation. This might seem fast on your terminal, but it becomes a critical bottleneck when:

STDOUT is piped to a slow consumer: Files on slow disks, network streams, terminals that can't keep up
Large log messages: Writing megabytes of data can take hundreds of milliseconds
High-frequency logging: Each log call blocks your thread, multiplying the cost
Performance-critical applications: Web servers, real-time systems, high-throughput data processing

The impact: Each log operation can take 1-10ms or more, during which your application is doing nothing but waiting for I/O to complete.

Problem #2: Rust's Standard Library Doesn't Support Non-Blocking STDOUT

Real-world example: When building Rust modules that run inside Node.js (using N-API, neon, or similar), Node.js sets STDOUT/STDERR to non-blocking mode by default. This causes standard Rust logging crates to panic with "Resource temporarily unavailable" errors, making them unusable in Node.js environments without special handling.

You might think: "I'll just set STDOUT to non-blocking mode at the OS level!" Unfortunately, this doesn't work with Rust's standard library:

// Set STDOUT to non-blocking mode (using fcntl)
set_nonblocking(stdout);

// This will PANIC when the buffer is full!
println!("Large message");  // L thread panicked: failed printing to stdout: Resource temporarily unavailable

The problem: When STDOUT/STDERR is in non-blocking mode, the OS returns WouldBlock errors when the output buffer is full. Rust's std::io::Stdout and the println!/eprintln! macros are not designed to handle this - they will panic immediately.

This happens with:

Rust modules running in Node.js: Node.js uses non-blocking I/O by default, causing panics in standard Rust crates
Large messages that don't fit in the kernel buffer (~64KB on most systems)
Parallel usage from multiple threads overwhelming the output buffer
Any situation where the consumer can't keep up with your write rate

You cannot simply use non-blocking I/O with Rust's standard library - you need proper handling of WouldBlock errors.

Problem #3: The Performance Impact

In a typical web application logging at INFO level:

// Each request logs ~5-10 times
log::info!("Request received: {}", request);
// ... blocked for 1-5ms ...
log::debug!("Processing: {}", data);
// ... blocked for 1-5ms ...
log::info!("Response sent: {}", response);
// ... blocked for 1-5ms ...

Result: N ms of your request latency is spent waiting for I/O operations. In a system handling 1000 req/s, this can be the difference between meeting your SLA and missing it.

Benchmarks

See BENCH_RESULTS for detailed results, metrics, and instructions.

License

MIT

Formating and options part are based on rust-simple_logger, which is authored by Sam Clements under MIT license

log_nonblock 0.1.7