Module controller 

The controller: the VM that runs the environment’s playbooks (provision:) on every target that runs playbooks (Vagrant, Proxmox, the clouds, isoloom provision), with one script, so they all retry the same way.

After a failed run, when no task failed on its own, the hosts that failed only because of the connection run the playbook again, on those hosts only (--limit @<playbook>.retry), after a pause, up to four runs in all:

• unreachable hosts (a Windows host dropping WinRM for a minute after a domain join or an update reboot);
• hosts whose task was cut by a dropped connection: a failure whose message is in the known list of dropped connections (WinRM’s shell crashing mid-task, an SSH session reset).

A task that failed on its own is real: the setup stops at once, nothing is retried. So does a run with no recap (a playbook that doesn’t parse). Every wait and run is logged, so the output explains what happened.

Functions

provision_script

The controller’s script running the environment’s playbooks, in order, with the inventory at /etc/isoloom/inventory.ini plus each step’s own inventories, as it runs when the environment comes up; isoloom provision runs it again on a running environment.