Skip to main content

Module buffer

Module buffer 

Source
Expand description

The on-disk ring buffer, the tombstone, and the wipe.

Everything lives under $CODEWHALE_HOME/telemetry/, created 0700, with every file 0600:

filerole
buffer.jsonlone JSON event per line, awaiting a flush
buffer.jsonl.locka sibling lock file; never the data file
dryrun.jsonlthe sink when the endpoint resolves empty, same ring policy
state.jsonlast version seen and last flush attempt
install_id.jsonthe random install id
disabledthe tombstone: present ⇒ nothing is appended, drained, or sent

Appends never take a lock. One O_APPEND write(2) under PIPE_BUF is atomic on every filesystem this ships to, and taking fd_lock here would be a blocking acquisition on the panic hook and the SIGINT path. flock is per-fd within a process, so an actor panic while holding the compaction lock would self-deadlock the hook — catch_unwind runs after the hook, so it cannot save this — and a second Codewhale process sharing CODEWHALE_HOME would hang Ctrl-C, breaking the second-signal contract in main.rs.

Compaction is the only lock holder and uses try_write(): on contention it skips this cycle. Appenders re-open per append, so a compaction rewrite cannot leave anyone writing to a stale inode.

Constants§

MAX_BYTES
Byte ceiling for either sink.
MAX_EVENTS
Newest events retained in either sink.
MAX_LINE_BYTES
A single append must fit in one atomic write(2).

Functions§

append
Append one serialized event or batch to path.
append_locked
Append a line that is too large for one atomic write(2), serialising against other writers with the compaction lock instead.
arm
Clear the tombstone and drop anything buffered before consent.
buffer_path
buffer.jsonl — the pending-event sink.
drain
Take every buffered line and truncate the buffer, under the compaction lock.
dryrun_path
dryrun.jsonl — where batches go when the endpoint resolves to None.
ensure_dir
Create the telemetry directory 0700, if it is missing.
install_id_path
install_id.json.
lock_path
buffer.jsonl.lock — the sibling lock file. Never the data file, and never unlinked: replacing it would leave appenders and compactors holding different inodes and serialising against nothing.
read_lines
Read every intact line from path, dropping a torn trailing line.
state_path
state.json.
tombstone_path
disabled — the tombstone.
tombstone_present
Whether the tombstone is present.
truncate
Truncate a file to zero length, leaving the inode in place. A missing file is not an error.
try_with_lock
Run operation holding the exclusive compaction lock if it is free.
wipe
Wipe every trace of collection, leaving a permanent tombstone.
with_lock
Run operation holding the exclusive compaction lock, blocking.