Module cgroup_memory 

cgroup v2 memory counters of a detached docker container (issue #182).

Docker’s State.OOMKilled is container-wide and sticky (moby/moby#43564): it says that some process in the container was OOM-killed at some point, but not how many, not whether the container hit its own --memory limit or the whole host ran out, and not how close the execution came to its limit. The kernel provides raw counters per cgroup (https://docs.kernel.org/admin-guide/cgroup-v2.html#memory-interface-files):

memory.events `oom`       allocation events reaching the memory limit
memory.events `oom_kill`  processes killed here by *any* OOM killer
memory.peak / memory.max  peak usage (Linux 5.19+) and limit (`max` when
                          unlimited)

These counters have different units and are hierarchical. A group OOM can kill several processes for one allocation event; earlier allocation events can also coexist with a later host OOM. Their comparison cannot establish container, parent or host scope (issue #185).

The cgroup is removed when the container stops, so the values cannot be read afterwards. The detached completion watcher therefore samples them while the container runs and hands the last sample to the post-mortem, the finalizer and the recovery step. Best effort by design: no cgroup v2, a remote docker daemon or a hidden /proc leave the counters unknown, and a kill in the last sampling interval can be missed when the cgroup is gone before the final read.

Modules

cgroup_shell_vars

Shell variables used by the sampler snippets.

Structs

CgroupMemory

The counters of one sample. limit_bytes is None without a limit.

Enums

OomScope

Where the OOM killer that killed processes in the container came from.

Constants

CGROUP_SAMPLE_BACKOFF_MAX_SECONDS

Longest wait between rounds that keep failing (issue #247), unless the interval itself is longer. Each round without a sample doubles the wait.

CGROUP_SAMPLE_INTERVAL_SECONDS

Default seconds between samples; the watcher environment may override it.

CGROUP_SAMPLING_OFF_REASON

Interval value that turns the sampler off (issue #242).

CGROUP_STREAM_NAME

$0 of the long-lived exec, visible in ps inside the task container.

CGROUP_UNREADABLE_EXIT

Exit status of both in-container reads for unreadable counters.

Functions

build_cgroup_functions_snippet

Shell functions shared by the sampler and its final read:

build_cgroup_memory_log_snippet

Shell fragment appending the Memory: line of the post-mortem when a sample exists. quoted_log_path must already be shell-quoted.

build_cgroup_sampler_start_snippet

Shell fragment starting the background sampler. It writes each sample to a temp file with mv, so the watcher never reads a half-written line, and ends by itself once the container is gone.

build_cgroup_sampler_stop_snippet

Shell fragment stopping the sampler once the container has exited and leaving the last sample (<max> <peak> <oom> <oom_kill>, or empty) in $__start_command_cgroup. The cgroup can outlive the main process for a moment, so it is read one last time when it still exists.

build_cgroup_stream_script

In-container loop of the long-lived sampler exec (issue #242): $1 is the cgroup directory. The host paces it with one newline per interval on stdin (docker exec -i), so it runs only shell builtins: the task’s own CPU limit pays for no cat, sed or sleep processes. Stdin reaches EOF when the exec’s client is gone, so the loop never outlives the sampler; dockerd would accept its output even then. Exit 3: unreadable counters (no delegated memory controller), which no later round can change.

describe_cgroup_oom_scope

Unknown for observed kills, None when no kills were observed.

deserialize_cgroup_memory

deserialize_with helper for ExecutionRecord::cgroup_memory.

format_cgroup_memory

Human-readable counters for --status, e.g. peak 255.9 MiB of 256.0 MiB limit, oom 0, oom_kill 3 (...).

format_cgroup_memory_log_line

The Memory: line for a sample, exactly as the watcher’s shell writes it. Used where Rust writes the post-mortem itself (the recovery step).

normalize_cgroup_memory

Normalize counters read back from an execution record. Lenient, like the JavaScript reader: a malformed value never makes the record unreadable.

parse_cgroup_memory_sample

Parse a sample written by the watcher (<memory.max> <memory.peak> <oom> <oom_kill>); None without a sample.