1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
/*!
# Proper daemonization and early startup
## What is daemonization
Traditionally, unix services are called daemons. When they start, they go into background instead
of staying in the terminal they were started from. That makes the terminal available for further use
and they don't get killed once the user terminates the session.
While there are alternatives nowdays (`systemd` keeps the services it starts in "foreground", at
least from the service's point of view, there's an external `daemonize` program, ...), it is still
expected "proper" services are able to go into background on their own and have this ability
configurable.
Daemonization, among other things, entails:
* Switching to either the `/` directory or to the daemon's home, to leave the place it was started
from.
* Possibly switching the user it runs under and dropping privileges.
* Closing `stdin`, `stdout` and `stderr`.
* Detaching from the terminal, by performing `fork` twice.
## The problem
There are some subtleties about the order things are done (some of the problems are general, some
are made even harder by the desire to go into background).
* Printing help (and similar things, like printing configuration) and terminating should happen
before the program attempts to actually do something. This lower the chance that something is
undesirable and also the chance it would fail and would not get around to the help.
* Configuring logging, panic handlers and similar should happen as early as possible, to preserve
the information if the startup fails. Specifically, the program should be logging to a file by
the time the program tries to go into background.
* Daemonization uses `fork` internally and that one is **not safe** to be performed in a
multi-threaded program. Therefore, daemonization needs to happen early enough ‒ if any kind of
initialization starts additional threads, this may be performed only after daemonization is
complete. Things known to start threads are `tokio` (or other worker threadpools and schedulers)
and background logging. In particular, it is not possible to perform daemonization in an
application created with the `#[tokio::main]` annotation and it is not possible to start
background logging before daemonization.
* Registering signal handlers is recommended before additional threads are started (though failing
to do that is not inherently UB as with the daemonization, it only contains a tiny race condition
when chaining multiple signal-handling libraries together).
## The correct order
While multiple solutions are probably possible, here we present one observed to work.
* Install the emergency shutdown signals handling (spirit handles signals and initiates a graceful
shutdown, but if it gets stuck, we want second termination signal to have an immediate effect).
* Handle the `--help`, `--help-config` and similar early, before any possible side effects or
fallible operations (unfortunately, the handlers need to parse configuration before
`--dump-config`, which might come a bit later).
* Configure first-stage logging. This one must happen without background logging (logging in a
background thread, which might be often more efficient, but we can't afford to do it just yet).
* Perform daemonization.
* Switch to full-featured logging, optionally with the background thread.
* Start all the other things, including ones that may need threads.
Beware that the pipelines and extensions are run in the order of registration (inside each relevant
phase ‒ before-config, config validation, confirmation of thereof, on-config), but these are still
delayed until the start of the `run` method. If you start any threads manually, do so within the
`run`.
## Example
```
use serde::{Deserialize, Serialize};
use spirit::{utils, Pipeline, Spirit};
use spirit::fragment::driver::SilentOnceDriver;
use spirit::prelude::*;
use spirit_cfg_helpers::{Opts as CfgOpts};
use spirit_daemonize::{Daemon, Opts as DaemonOpts};
use spirit_log::{Cfg as Logging, Opts as LogOpts, CfgAndOpts as LogBoth};
use spirit_log::background::{Background, OverflowMode};
use structdoc::StructDoc;
use structopt::StructOpt;
#[derive(Clone, Debug, StructOpt)]
struct Opts {
#[structopt(flatten)]
daemon: DaemonOpts,
#[structopt(flatten)]
logging: LogOpts,
#[structopt(flatten)]
cfg_opts: CfgOpts,
// Other stuff goes in here
}
impl Opts {
fn logging(&self) -> LogOpts {
self.logging.clone()
}
fn cfg_opts(&self) -> &CfgOpts {
&self.cfg_opts
}
}
#[derive(Clone, Debug, Default, Deserialize, StructDoc, Serialize)]
struct Cfg {
#[serde(default)]
daemon: Daemon,
#[serde(default, skip_serializing_if = "Logging::is_empty")]
logging: Logging,
// Other stuff
}
impl Cfg {
fn logging(&self) -> Logging {
self.logging.clone()
}
}
fn main() {
// Set the emergency signal shutdown
utils::support_emergency_shutdown().expect("Installing signals isn't supposed to fail");
Spirit::<Opts, Cfg>::new()
// Some extra config setup, like `.config_defaults() or `.config_env`
// Put help options early. They may terminate the program and we may want to do it before
// daemonization or other side effects.
.with(CfgOpts::extension(Opts::cfg_opts))
// Early logging without the background thread. Only once, then it is taken over by the
// pipeline lower. We can't have the background thread before daemonization.
.with(
Pipeline::new("early-logging")
.extract(|opts: &Opts, cfg: &Cfg| LogBoth {
cfg: cfg.logging(),
opts: opts.logging(),
})
// Make sure this is run only once, at the very beginning.
.set_driver(SilentOnceDriver::default()),
)
// Plug in the daemonization configuration and command line arguments. The library will
// make it alive ‒ it'll do the actual daemonization based on the config, it only needs to
// be told it should do so this way.
//
// Must come very early, before any threads are started. That includes any potential
// logging threads.
.with(unsafe {
spirit_daemonize::extension(|cfg: &Cfg, opts: &Opts| {
(cfg.daemon.clone(), opts.daemon.clone())
})
})
// Now we can do the full logging, with a background thread.
.with(
Pipeline::new("logging")
.extract(|opts: &Opts, cfg: &Cfg| LogBoth {
cfg: cfg.logging(),
opts: opts.logging(),
})
.transform(Background::new(100, OverflowMode::Block)),
)
// More things and pipelines can go in here, including ones that start threads.
.run(|_| {
// And we can start more threads here manually.
Ok(())
});
}
```
*/