Expand description
Launch-time recovery for killed detached docker sessions (issue #176).
$ --isolated docker --detached --on-kill-resume 3 --recovery-command B -- A
runs A; when the main process is killed (exit 137, or OOMKilled with no
exit status of its own), the detached completion watcher hands the inspected
facts to this module, which restarts the same container so B continues
on the same filesystem, with the same resource limits (they live in the
container’s HostConfig), under the same execution UUID and appending to the
same log file.
How B replaces A inside the same container: a container launched with a
recovery command runs a tiny selector as its command. The selector runs A
normally, but runs B when the marker file exists. Before docker start,
the marker (holding the attempt number) is copied into the stopped
container with docker cp. Without a recovery command the original command
is simply started again.
Like detached_finalize, the entry point runs as a short-lived child of the
watcher on a host whose CLI invocation is long gone, so it never panics on
bad input and never prints anything.
Structs§
- Recovery
Delay Hooks - Where the
--on-kill-resume-delaywait gets its randomness and sleeps; tests swap both out (issue #181). - Recovery
Facts - The docker facts the watcher hands over.
- Recovery
Outcome - Outcome of a recovery attempt;
recoveredis true only when the container was started again. - Recovery
Separator - What the
[Recovery k/N]separator reports.
Constants§
- INTERNAL_
RECOVER_ FLAG - Hidden argument the detached watcher re-invokes this binary with.
- KILLED_
EXIT_ CODE - Exit code of a process killed with SIGKILL (the OOM killer’s signal).
- RECOVERY_
ATTEMPT_ ENV - Environment variable carrying the attempt number into the recovery command.
- RECOVERY_
MARKER_ PATH - File whose presence makes the selector run the recovery command.
Functions§
- build_
recovery_ selector_ args - Wrap the container command so a later
docker startcan switch to the recovery command. - build_
recovery_ snippet - Shell condition run by the completion watcher: true when the container was
killed and the recovery entry point resumed it. Mirrors
is_killed_exit:OOMKilledonly counts without a non-negative exit code (issue #178). - format_
recovery_ separator - The
[Recovery k/N]separator written into the session log. - is_
killed_ exit - Whether the inspected facts describe a killed main process.
- recover_
killed_ execution - Resume a killed execution in its own container.
- recover_
killed_ execution_ with_ delay recover_killed_executionafter the random--on-kill-resume-delaywait (issue #181), with injectable randomness and sleep.- recovery_
selector - POSIX selector run as the container command when a recovery command is set:
sh -c SELECTOR start-command <shell> <flag> <recovery> <main argv...>. - run_
internal_ recover - Entry point for
--internal-recover-detached-docker <uuid> <exit> <oom> <started> <finished> <error> <cgroup>. Returns 0 only when the container was resumed; any other outcome lets the watcher continue with its normal cleanup, footer and finalization. - start_
system_ watcher - The real watcher: a detached shell following the resumed container.
Type Aliases§
- Watcher
Starter - Starts the completion watcher that follows a resumed container.