1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
//! Fragments of configuration.
//!
//! There are several crates that provide common fragments of configuration to configure some
//! specific functionality. These fragments are described in terms of the [`Fragment`] trait.
//!
//! # How to use them
//!
//! A [`Fragment`] can be used directly, usually through [`create`] method. The trait has some
//! other methods and associated types, but these are usually used only internally.
//!
//! The other option is to use fragments through [`Pipeline`]s. A pipeline describes how the
//! fragment is extracted from the configuration, how instances of resources it describes are
//! cached and created, how to post-process them and how to install or activate them. Depending on
//! the kind of fragment, less or more tweaking may be needed or desirable.
//!
//! # How pipelines work
//!
//! A [`Pipeline`] manages certain fragment of configuration and creates [`Resource`]s out of it.
//! There are several traits in play there.
//!
//! * A pipeline is triggered whenever new configuration is loaded. It runs its configured
//! [`Extractor`]. This extractor provides an instance of the [`Fragment`].
//! * A [`Driver`] is provided with the extracted fragment. The [`Driver`] decides if the
//! [`Resource`] needs to be recreated or if an old instance need to be destroyed. Each fragment
//! has its default [`Driver`], but the driver of a pipeline can be manually replaced by other
//! one, to for example change caching strategy. Note that the [`Driver`] is allowed to partition
//! the [`Fragment`] into smaller [`Fragment`]s and drive the rest of the pipeline multiple times
//! on the smaller ones (eg. a `Vec<F>` would be split into multiple runs over `F`).
//! * When creating the [`Resource`], there are two stages. First, a [`Seed`] is created with the
//! [`make_seed`] method. Then one or more [`Resource`]s are made out of the [`Seed`] with the
//! [`make_resource`] method. This allows more flexible caching strategies and allows for example
//! to change attached configuration without closing and opening a network socket if the port
//! haven't changed (which would be problematic, as spirit first tries to create the new instance
//! and only if it works gets rid of the old one ‒ but that couldn't be done if we still had the
//! old with the same port). Some [`Fragment`]s don't need this two-stage configuration,
//! therefore they have the [`Seed`] set to `()` and trivial [`make_seed`] method. You can use
//! the [`simple_fragment`](../macro.simple_fragment.html) macro to generate such trait
//! configuration.
//! * Then the resource goes through configured set of [`Transformation`]s. These may be quite
//! arbitrary, but they usually tie the resource with some kind of functionality ‒ a network
//! socket is provided with a function to handle each new connection, a HTTP server is provided
//! with the service it'll serve, etc. A freshly created [`Pipeline`] has no transformations, but
//! they can be added.
//! * Finally, the resulting product is installed using the [`Installer`]. Some fragments come with
//! a default installer, some do not. The [`Installer`] is often set as part of a transformation.
//! Nevertheless, an installer can always be set manually.
//! * The installer returns [`UninstallHandle`]s. These represent the lifetime of the installed
//! resources. The pipeline stores them until the time is right to destroy the resources ‒ then
//! it drops the handles, which results in removal of the resources.
//!
//! # Names
//!
//! Most methods around the mentioned traits take a `name: &'static str` parameter. This is used by
//! them to enrich log messages, as there might be multiple distinct parts of configuration of the
//! same type.
//!
//! The name is provided when creating the [`Pipeline`]. It needs to be a string literal, but as
//! this should correspond to specific functionality in the program, this should not be very
//! limiting.
//!
//! TODO: An example
//!
//! # How to create a fragment
//!
//! First, try to do it manually, without fragments or pipeline ‒ eg. write the code that takes the
//! configuration and creates something out of it and activates it.
//!
//! Then decide how this should be reloaded when new configuration appears, how it can be
//! reinstalled or if and how it should be cached.
//!
//! Then you can have a look at available pieces, like ready-made
//! [drivers][crate::fragment::driver] or installers. Sometimes, they come from another trait ‒ eg.
//! the [`spirit_tokio`] crate comes with an installer for futures. Usually, you need to implement
//! only the [`Fragment`] trait (either in two-stage or single-stage fashion), but sometimes you
//! might need to add some kind of [`Transformation`] to tie the part that comes from the
//! configuration with some actual code.
//!
//! You may also want to implement the [`Stackable`] and possibly [`Comparable`] traits for the
//! fragment.
//!
//! [`Fragment`]: crate::fragment::Fragment
//! [`Driver`]: crate::fragment::driver::Driver
//! [`create`]: crate::fragment::Fragment::create
//! [`Pipeline`]: crate::fragment::pipeline::Pipeline
//! [`Extractor`]: crate::fragment::Extractor
//! [`Resource`]: crate::fragment::Fragment::Resource
//! [`Seed`]: crate::fragment::Fragment::Seed
//! [`make_seed`]: crate::fragment::Fragment::make_seed
//! [`make_resource`]: crate::fragment::Fragment::make_resource
//! [`simple_fragment`]: crate::simple_fragment
//! [`Transformation`]: crate::fragment::Transformation
//! [`Installer`]: crate::fragment::Installer
//! [`UninstallHandle`]: crate::fragment::Installer::UninstallHandle
//! [`Stackable`]: crate::fragment::Stackable
//! [`Comparable`]: crate::fragment::driver::Comparable
//! [`spirit_tokio`]: https://docs.rs/spirit-tokio
use ;
use ;
use trace;
use DeserializeOwned;
use StructOpt;
use ;
use crate Extensible;
use crate AnyError;
/// An entity that is able to install a resource.
///
/// At the end of a [`Pipeline`][crate::fragment::pipeline::Pipeline] there's an installer. It
/// takes the (transformed) resource and somehow makes it active in the program.
///
/// An installer can be even a storage provided by a user where the resource is stored ‒ eg. a
/// proxy object to the resource where it can be switched.
///
/// Note that installation of the resource must not fail.
/// A sequence installer.
///
/// This is an auxiliary installer wrapper, to install instances from a collection of fragments.
///
/// Usually, this is used behind the scenes in things like `Vec<F: Fragment>` and shouldn't have to
/// be used by user directly.
/// A trait to mark [`Fragment`]s that can form collections.
///
/// If it makes sense to use collections of the fragment (eg. `Vec<F>` or `HashSet<F>`) in the
/// configuration, in addition to implementing the [`Fragment`] trait, mark the fragment by this
/// trait. Then a default implementation of [`Fragment`] will be provided for the common
/// collections.
/// A trait similar to [`Stackable`], but marking the ability to be optional.
///
/// This allows using the [`Fragment`] as `Option<F>`. This is automatically implemented for all
/// [`Stackable`] fragments.
/// A fragment of configuration.
///
/// The fragment is the part of configuration [`Pipeline`][pipeline::Pipeline]s work with. It
/// usually comes directly from the configuration, but it also may be constructed by the
/// [`Extractor`] (maybe by combining parts of the configuration, or combining data from
/// configuration and command line options).
///
/// See the details of how [pipelines work](index.html#how-pipelines-work).
///
/// Note that fragments as described by this trait create their resources in two stages ‒ first a
/// [`Seed`] is created, then turned into [`Resource`]. The [`Driver`] controls this. Note that one
/// [`Seed`] may be used to create multiple instances of the [`Resource`] (depending on the driver
/// either in parallel or sequentially, replacing the previous ones). However, if fragment doesn't
/// want to have this two-phase creation, it can set the [`Seed`] to `()`.
///
/// [`Seed`]: Fragment::Seed
/// [`Resource`]: Fragment::Resource
// TODO: Export the macro for other containers?
// TODO: The where-* should be where-?
}
}
}
fragment_for_seq!;
fragment_for_seq!;
fragment_for_seq!;
fragment_for_seq!;
fragment_for_seq!;
fragment_for_seq!;
/// A helper macro to implement a simple [`Fragment`].
///
/// The full implementation of a [`Fragment`] requires a lot of work that is not usually needed.
///
/// In case the [`Fragment`] should not implement the two-stage creation, this can be used to cut
/// down on the boilerplate a bit.
///
/// # Examples
///
/// ```rust
/// use spirit::AnyError;
/// use spirit::simple_fragment;
/// use spirit::fragment::Installer;
///
/// // The message as a resource
/// struct Message(String);
///
/// // An installer of a resource
/// #[derive(Default)]
/// struct MessageInstaller;
///
/// impl<O, C> Installer<Message, O, C> for MessageInstaller {
/// type UninstallHandle = ();
/// fn install(&mut self, message: Message, _name: &str) {
/// println!("{}", message.0);
/// }
/// }
///
/// // Configuration of a message from the config file
/// # #[allow(dead_code)] // Allow not using this structure.
/// struct MessageCfg {
/// msg: String,
/// }
///
/// simple_fragment! {
/// impl Fragment for MessageCfg {
/// type Resource = Message;
/// type Installer = MessageInstaller;
/// fn create(&self, _name: &'static str) -> Result<Message, AnyError> {
/// Ok(Message(self.msg.clone()))
/// }
/// }
/// }
/// # fn main() {}
/// ```
///
/// If the [`Driver`] is not provided (as in the above example), the [trivial
/// driver][crate::fragment::driver::Trivial] is going to be used.
///
/// [`Fragment`]: crate::fragment::Fragment
/// [`Driver`]: crate::fragment::driver::Driver
) => ;
=>
}
// TODO: How do we stack maps, etc?
// TODO: Arcs, Rcs, Mutexes, refs, ...
/// A trait describing something that extracts a fragment from configuration and command line
/// options.
///
/// The extractor is used every time a [`Pipeline`] is triggered, to get the
/// fragment out.
///
/// Usually, an extractor is a closure, but something else can implement the trait too.
///
/// Users usually don't need to interact with this trait directly.
///
/// Note that the extractor is lifetime-parametric. Usually the real extractor implements the trait
/// for all lifetimes (and it is not useful otherwise). This allows returning references into the
/// configuration, the [`Pipeline`] is able to work with that (given new enough `rustc` ‒ there
/// were some bugs preventing it from working; if that's the case, you can clone and return owned
/// values).
///
/// [`Pipeline`]: pipeline::Pipeline
/// A transformation of resources.
///
/// A [`Pipeline`] can contain a transformation of the [`Resource`] produced by the [`Fragment`].
/// This trait describes a transformation.
///
/// Note that transformations in a pipeline are usually composed together.
///
/// The transformation is also allowed (and sometimes required) to either transform or replace the
/// [`Installer`] of the [`Pipeline`]. The old [`Installer`] might not be able to install the new
/// [`Resource`] (since the type can change during the transformation) or might not even exist.
///
/// A transformation can be quite arbitrary thing that simply takes the old resource and produces
/// something new. But more often that not, it is used to somehow make a „dead“ resource (eg. a
/// network socket) „alive“ (eg. wrap it into a future that is then installed) ‒ or, in other
/// words, to tie the resource to some functionality.
///
/// It is also possible to use a transformation to post-process and tweak the resource a bit (add
/// more loggers, change a bit of configuration of the resource, ...).
///
/// # Type parameters
///
/// * `InputResource` is the resource on the transformation input; this one will be changed every
/// time the transformation is called.
/// * `InputInstaller` is the original installer that was present in the pipeline before the
/// transformation got added. Note that not all installers at that point are able to install the
/// `InputResource`. But if it is able, it can be used to delegate from the
/// [`OutputInstaller`][Transformation::OutputInstaller] (or just use it vanilla).
/// * `SubFragment` is the fragments the [`Transformation`] will be used on. Each
/// [`Resource`][Fragment::Resource] is always accompanied by the [`Fragment`] it came from. Note
/// that this might be some sub-part of the original [`Fragment`], as the [`Driver`] is allowed
/// to cut it into smaller pieces.
///
/// [`Resource`]: Fragment::Resource
/// [`Pipeline`]: pipeline::Pipeline