Skip to main content

Module logqueue

Module logqueue 

Source
Expand description

The console queue: the log lines gunnar writes to stderr go through ONE bounded queue and ONE drainer thread, so no request ever waits for the console to accept a byte.

§Why (MEASURED 2026-08-30)

On the appliance stderr is /dev/console = ttyS0, a 16550 UART that the hypervisor paces at its declared 115200 baud — about 30 KB/s on the KVM rig. A push narrates ~18 INFO lines of ~370 bytes each through the ssh.connection{…}:git.transfer{…} span prefix, and every one of them was written synchronously from the connection’s own thread: tracing’s fmt layer holds the writer while the kernel’s serial driver waits for the transmit register byte by byte. On the rig that was 8–50 ms per log line, between lines with no work in between; a push-create that the same binary serves in 3 ms on a developer box took 161–228 ms, and eight parallel clients got 6–7.6 ops/s because the UART is one device and every thread queued on it. Stock git, with no console to talk to, created repositories at 103 ops/s. The serial log’s own byte rate over the phase was 32.0 KB/s — the console was the clock.

§What this does

The fmt layer formats each event into a per-event buffer (the MakeWriter contract: one writer per event, dropped when the event is written). On drop the line is handed to a sync_channel of CAPACITY lines and the thread carries on. A single drainer thread owns stderr and writes lines in order.

  • WARN and ERROR block until the queue has room — never dropped. The audit trail (gunnar::audit) is WARN, the dashboard’s alert rows are WARN, and one such line waiting a few milliseconds behind a backlog is the right trade; a lost one is not.
  • INFO and below are dropped when the queue is full, and counted. The drainer says so on the console itself — event="log.dropped" with the count — the next time it catches up, so a sampled console never reads as a complete one. The dashboard is unaffected: it reads the in-process tap (crate::logtap), not the console.

The queue is used when stderr is not a terminal: PID 1 on the appliance, a systemd unit under journald, a test harness with RUST_LOG set. A human at a terminal keeps the synchronous writer: there the console is fast, and println! output interleaving with the log in order matters more than a few microseconds per line.

§Exit

A process that returns from main takes whatever is still queued with it, so the two entry points call flush on their way out. It waits, bounded, for the drainer to empty the queue.

Structs§

LineWriter
One formatted event. Bytes collect here; the line is queued on drop.
QueueWriter
The MakeWriter half: hands the fmt layer one LineWriter per event.

Constants§

CAPACITY
Lines the queue holds before INFO starts being dropped. At ~370 bytes a line this is ~1.5 MB at the very worst, preallocated by the channel once.

Functions§

active
Whether the queue is installed (stderr was not a terminal at init).
console_sink
What the drainer writes to: the console set_console named, else stderr.
console_write
Write text straight to the console (a PID 1 shout), not through fd 2: with stderr captured, fd 2 is the pipe and the line would be tapped twice.
dropped
Total INFO-and-below lines the console could not carry so far.
flush
Wait, at most timeout, for every queued line to reach the console.
install
Start the drainer and hand back the MakeWriter the fmt layer takes.
set_console
Name console as the device the drainer writes to. Before crate::logging::init; the first caller wins.
set_tap
Hand every console line to f as well. The first caller wins.
tap_line
A line said OUTSIDE tracing (the boot narrator), handed to the tap too.
written
Total lines written to the console so far.