shep 0.18.0

The shep binary: a process manager that keeps a flock of long-running processes alive on macOS, Linux and Windows, with logs, watch and cron restarts, and webhook alerts
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
//! One proposed dog table, asked whether the schema that describes it takes
//! it: [`table_refusal`].
//!
//! The door [`super::refusal`] walks one keystroke through is the same door a
//! write needs before it sends a whole table, and a table is what
//! `Request::SetDogConfig` and `Request::SetSheepDogSettings` carry. This
//! half answers the table: it takes the schema a dog answered (`--schema` for
//! its own section, the `x-shep-sheep` value for its table on a sheep) and
//! the table about to be written, and answers the first refusal it finds,
//! with the key path the value sits at and the reason for it. [`table_refusal`]
//! covers every key the schema names; [`touched_refusal`] covers only the keys
//! its caller chose, with the same arms, the same order and the same sentences.
//!
//! Every rule is the one [`super`] already applies to a typed buffer, read
//! off the same [`crate::lookout::field::flattened`] walk of the schema: the
//! same bounds, the same two shep grammars, the same `pattern`, the same item
//! schema for an element of a list, the same arms of a tagged union, and the
//! same nested tables flattened to dotted leaves. A value cannot pass one
//! door and fail the other, which is the whole reason there is one module
//! rather than a copy in the pane and a copy in the CLI.
//!
//! One rule belongs to a table alone, because a table is JSON and a buffer is
//! text: a value has to be the JSON type the schema names before any rule on
//! it is read. `keep = "8"` in a dog's section is a string, and the pane's
//! door would never have filed one for an `integer`; handing it to the same
//! parse as text would take it. So each arm asks what kind of value it was
//! given, and refuses with the sentence the pane already uses for a value of
//! the wrong kind.
//!
//! Two things it does not do, both because shep is not the authority on a
//! dog's config beyond what that dog published:
//!
//! - It refuses nothing a schema says nothing about. A key the schema does
//!   not name rides along through a write untouched, a map of arbitrary keys
//!   is taken as it comes, and a read-only row ([`FieldKind::Opaque`], which
//!   is what a nullable item list and a table that holds itself flatten to)
//!   is not checked.
//! - It refuses nothing for a key the table does not carry, with one
//!   exception: a field the arm a tagged union names lists as required. An
//!   absent key and a `null` both mean unset, which is the pane's own unset
//!   spelling and, for a dog, the way a value goes back to the dog's default.

use std::collections::BTreeSet;

use serde_json::{Map, Value};

use crate::lookout::field::{
    Entries, Field, FieldKind, Flattened, ListItem, Union, flattened, segment,
};

use super::numbers::{integer_refusal, number_refusal};
use super::{
    item_refusal, not_a_number, not_a_whole_number, refusal, required_field, shown_value,
    takes_one_of,
};

/// Why a proposed table is not one the dog's schema takes.
///
/// `path` is the property path the value sits at, one segment per table, so
/// a caller can name the key it came from even when the sentence spells it
/// dotted. `text` is the sentence, built by the same arms [`super::refusal`]
/// and [`super::item_refusal`] build theirs.
///
/// `Debug` is derived (IR-41), and safe for the same reason [`super::Refusal`]'s
/// is: every sentence here resolves what it may print through
/// [`crate::dog_settings::shown_value`] before it builds one, so a secret's
/// value never reaches `text`. Pinned by an exact-string test.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct TableRefusal {
    /// The key path, from the top of the table down, in property names.
    pub path: Vec<String>,
    /// The sentence to print.
    pub text: String,
}

impl TableRefusal {
    fn at(path: &[String], text: String) -> Self {
        Self {
            path: path.to_vec(),
            text,
        }
    }
}

/// Which keys of the proposed table the walk answers for.
///
/// [`Scope::Whole`] is what a caller that chose every key needs.
/// [`Scope::Touched`] is what a pane needs: it writes a whole table, but the
/// only keys it chose are the ones an operator edited, and a value the
/// operator wrote into `dogs.toml` or the Flockfile by hand goes back out as
/// it came. Whether lookout should ever refuse one is a separate decision.
#[derive(Clone, Copy)]
enum Scope<'a> {
    /// Every key the schema names.
    Whole,
    /// Only the keys whose property path is in `touched`.
    Touched(&'a BTreeSet<Vec<String>>),
}

impl Scope<'_> {
    fn holds(self, path: &[String]) -> bool {
        match self {
            Scope::Whole => true,
            Scope::Touched(touched) => touched.contains(path),
        }
    }
}

/// The refusal for `path`, or [`None`] when `scope` does not cover that key.
fn report(scope: Scope<'_>, path: &[String], text: String) -> Option<TableRefusal> {
    scope.holds(path).then(|| TableRefusal::at(path, text))
}

/// Why `table` is not a table `schema` takes, or [`None`] when nothing in it
/// disagrees with the schema.
///
/// `schema` is the whole answer, not a `properties` map: a dog's own section
/// schema, or its `x-shep-sheep` value with the root's `$defs` attached, the
/// same shape [`crate::lookout::pane::ConfigPane::dog`] and
/// [`crate::lookout::pane::ConfigPane::sheep_dog`] build their rows from. The
/// walk is [`crate::lookout::field::flattened`], so the leaves this checks
/// are the leaves the pane edits, one per row, dotted for a nested table.
///
/// The first refusal in row order, not every one: an operator fixes the key
/// the sentence names and reads what comes back, and a schema that disagrees
/// with a table in four places has four sentences to be told in turn.
///
/// [`None`] is not a promise that the dog accepts the table. It says no rule
/// the schema published is broken by it, which is all shep can know about a
/// config its author wrote in another binary.
#[must_use]
pub fn table_refusal(schema: &Value, table: &Map<String, Value>) -> Option<TableRefusal> {
    flattened_refusal(&flattened(schema), table, &[], "", Scope::Whole)
}

/// [`table_refusal`] over only the keys in `touched`.
///
/// `touched` holds property paths, the same paths [`crate::lookout::field::flattened`]
/// built the row keys from: `["models", "provider"]` for a nested table's
/// leaf, `["clients", "0", "name"]` for one field of one entry of an array of
/// tables. A refusal the walk finds at any other key is not reported, so a
/// caller that writes a table it only partly chose asks about the part it
/// chose. Nothing else changes: the same arms, the same order, the same
/// sentences.
///
/// An unreported refusal does not end the walk. A bad value the caller did not
/// choose rides along in the write either way, and stopping there would hide
/// the refusal at the key it did choose, which is the one it can act on.
#[must_use]
pub fn touched_refusal(
    schema: &Value,
    table: &Map<String, Value>,
    touched: &BTreeSet<Vec<String>>,
) -> Option<TableRefusal> {
    flattened_refusal(&flattened(schema), table, &[], "", Scope::Touched(touched))
}

/// Every leaf of `flat`, read out of `table`, in row order.
///
/// `base_path` and `base_key` are where `table` sits inside the table the
/// caller started at: empty at the top, and one entry deep for each table of
/// an [`FieldKind::Entries`] row. A shape's own `shapes` is empty, so this
/// descends one entry deep and stops.
fn flattened_refusal(
    flat: &Flattened,
    table: &Map<String, Value>,
    base_path: &[String],
    base_key: &str,
    scope: Scope<'_>,
) -> Option<TableRefusal> {
    for field in flat.fields.fields() {
        let Some(rest) = flat.paths.get(&field.key) else {
            continue;
        };
        if let Some(refused) = leaf_refusal(flat, field, rest, base_path, base_key, table, scope) {
            return Some(refused);
        }
    }
    None
}

/// One leaf row, and the value `table` holds at `rest`.
///
/// `rest` is the row's own path below `base_path`, which is what both the
/// lookup and the sentence key are built from: [`crate::lookout::field::flattened`]
/// built `field.key` out of exactly those segments, quoting any of them a dot
/// would split. `base_path` and `base_key` are where `table` sits in the table
/// the caller started at, and only the reported path and key carry them: a
/// nested table is looked up from its own top.
fn leaf_refusal(
    flat: &Flattened,
    field: &Field,
    rest: &[String],
    base_path: &[String],
    base_key: &str,
    table: &Map<String, Value>,
    scope: Scope<'_>,
) -> Option<TableRefusal> {
    let path: Vec<String> = base_path.iter().chain(rest.iter()).cloned().collect();
    let key = below(base_key, &field.key);
    match held(table, rest) {
        Held::Unset => None,
        // A scalar where the schema has a table: every leaf below it reads
        // as unset, so the key the operator has to fix is this one.
        Held::Blocked { value, depth } => {
            let prefix: Vec<String> = rest
                .iter()
                .take(depth + 1)
                .map(|name| segment(name))
                .collect();
            report(
                scope,
                &base_path
                    .iter()
                    .chain(rest[..=depth].iter())
                    .cloned()
                    .collect::<Vec<_>>(),
                not_a_table(
                    &below(base_key, &prefix.join(".")),
                    &shown_for(field.secret, value),
                ),
            )
        }
        Held::Leaf(value) => match &field.kind {
            FieldKind::Entries(kind) => {
                entries_refusal(flat, field, kind, value, &path, &key, scope)
            }
            FieldKind::Union(union) => union_refusal(union, value, &path, &key, scope),
            _ => scalar_refusal(field, value, &key, &path, scope),
        },
    }
}

/// `value` against `field`, as a refusal naming `key` and the path `path` it
/// sits at, for every kind that holds a value rather than a table of its own.
///
/// The arms are [`super`]'s, with the row's key replaced by the full path:
/// [`integer_refusal`] and [`number_refusal`] for the two numeric kinds,
/// [`refusal`] for a string's grammar or `pattern`, and [`item_refusal`] for
/// one element of a list, against the bounds its item schema published.
///
/// Each arm is reached through the value's JSON type first, and a value of
/// the wrong one is refused with [`not_a_whole_number`], [`not_a_number`],
/// [`not_a_string`] or [`not_a_list`] rather than handed to a parse that
/// would read it as text.
fn scalar_refusal(
    field: &Field,
    value: &Value,
    key: &str,
    path: &[String],
    scope: Scope<'_>,
) -> Option<TableRefusal> {
    let shown = shown_for(field.secret, value);
    match &field.kind {
        // The parse is for a value that is a number. A JSON string is not one,
        // however it reads: `keep = "8"` is a string in the table the section
        // writer hands on, and the pane files an integer as `Value::from(i64)`
        // and a decimal as `Value::from(f64)`, never as a string.
        FieldKind::Integer if !value.is_number() => {
            report(scope, path, not_a_whole_number(key, &shown).text)
        }
        FieldKind::Integer => integer_refusal(key, &field.bounds, &text_of(value), &shown)
            .and_then(|r| report(scope, path, r.text)),
        FieldKind::Number if !value.is_number() => {
            report(scope, path, not_a_number(key, &shown).text)
        }
        FieldKind::Number => number_refusal(key, &field.bounds, &text_of(value), &shown)
            .and_then(|r| report(scope, path, r.text)),
        FieldKind::Bool => value
            .as_bool()
            .is_none()
            .then(|| not_a_toggle(key, &shown))
            .and_then(|text| report(scope, path, text)),
        FieldKind::Text | FieldKind::Suggested(_) => match value.as_str() {
            Some(text) => {
                refusal(&named(field, key), text).and_then(|r| report(scope, path, r.text))
            }
            None => report(scope, path, not_a_string(key, &shown)),
        },
        // A closed set on a scalar is chosen from rather than typed, so it
        // never reaches [`refusal`] at all. A table can hold anything.
        FieldKind::Choice(names) => match value.as_str() {
            Some(text) if names.iter().any(|name| name == text) => None,
            _ => report(scope, path, takes_one_of(key, &shown, names).text),
        },
        FieldKind::List(item) => {
            let Some(elements) = value.as_array() else {
                return report(scope, path, not_a_list(key, &shown));
            };
            let bounds = field.bounds.items.as_deref().cloned().unwrap_or_default();
            for (index, element) in elements.iter().enumerate() {
                // A `null` element is the unset spelling, not a value to
                // hold to a rule. A list whose items may be `null` is not a
                // list row in the first place: it flattens to one read-only
                // row, and this arm never sees it.
                if element.is_null() {
                    continue;
                }
                let slot = below(key, &index.to_string());
                let element_path = below_path(path, &index.to_string());
                // The element's own type before its item schema's rules, for
                // [`FieldKind::Integer`]'s reason: [`text_of`] would hand `8`
                // to a string list's `pattern` as "8" and `true` as "true",
                // and `ListPane::apply_typing` files an integer element as
                // `Value::from(i64)`.
                let shown_element = shown_for(field.secret, element);
                // An element this caller did not choose is not an answer to
                // give, and not a reason to stop looking at the ones it did:
                // the bad element rides along in the write either way, so the
                // walk only stops on a refusal it is allowed to report.
                let wrong_type = match item {
                    ListItem::Integer if !element.is_number() => {
                        Some(not_a_whole_number(&slot, &shown_element).text)
                    }
                    ListItem::Text | ListItem::Choice(_) if element.as_str().is_none() => {
                        Some(not_a_string(&slot, &shown_element))
                    }
                    ListItem::Integer | ListItem::Text | ListItem::Choice(_) => None,
                };
                if let Some(refused) =
                    wrong_type.and_then(|text| report(scope, &element_path, text))
                {
                    return Some(refused);
                }
                let item_rule = item_refusal(&slot, item, &bounds, &text_of(element), field.secret)
                    .and_then(|refused| report(scope, &element_path, refused.text));
                if let Some(refused) = item_rule {
                    return Some(refused);
                }
            }
            None
        }
        // A map of arbitrary keys, a table this pane cannot walk, and a
        // read-only row each hold whatever the schema said nothing about.
        FieldKind::Map | FieldKind::Opaque | FieldKind::Entries(_) | FieldKind::Union(_) => None,
    }
}

/// An array or a map of tables: every entry checked against the item schema
/// [`crate::lookout::field::flattened`] flattened beside the row, the same
/// shape the pane draws an entry's rows from.
fn entries_refusal(
    flat: &Flattened,
    field: &Field,
    kind: &Entries,
    value: &Value,
    path: &[String],
    key: &str,
    scope: Scope<'_>,
) -> Option<TableRefusal> {
    let shape = flat.shapes.get(&field.key)?;
    let shown = shown_for(field.secret, value);
    match kind {
        Entries::Array => {
            let Some(entries) = value.as_array() else {
                return report(scope, path, not_a_list(key, &shown));
            };
            for (index, entry) in entries.iter().enumerate() {
                let slot = below(key, &index.to_string());
                let entry_path = below_path(path, &index.to_string());
                // An entry this caller did not choose is skipped, not reported,
                // and the entries after it are still checked.
                let Some(table) = entry.as_object() else {
                    let refused = not_a_table(&slot, &shown_for(field.secret, entry));
                    if let Some(refused) = report(scope, &entry_path, refused) {
                        return Some(refused);
                    }
                    continue;
                };
                if let Some(refused) = flattened_refusal(shape, table, &entry_path, &slot, scope) {
                    return Some(refused);
                }
            }
            None
        }
        Entries::Map => {
            let Some(entries) = value.as_object() else {
                return report(scope, path, not_a_table(key, &shown));
            };
            for (name, entry) in entries {
                let slot = below(key, &segment(name));
                let entry_path = below_path(path, name);
                let Some(table) = entry.as_object() else {
                    let refused = not_a_table(&slot, &shown_for(field.secret, entry));
                    if let Some(refused) = report(scope, &entry_path, refused) {
                        return Some(refused);
                    }
                    continue;
                };
                if let Some(refused) = flattened_refusal(shape, table, &entry_path, &slot, scope) {
                    return Some(refused);
                }
            }
            None
        }
    }
}

/// A tagged union: the table has to name one of the arms, every field of the
/// arm it names is held to its own bounds, and the arm's own `required` list
/// is enforced.
///
/// A required field the table does not hold is the one refusal this walk gives
/// for a key that is absent, and it is reported at the union rather than at
/// the field: the key the operator has to fix is the table that holds no value
/// for it. A field this pane has no editor for is not asked for, the same rule
/// [`crate::lookout::pane::ConfigPane`] applies to the pane's filed edits
/// through `incomplete_union`, which builds its sentence through
/// [`required_field`] as this door does.
///
/// A union is never a secret row: a schema that reaches a secret is read as
/// no [`crate::lookout::field::Union`] at all, so nothing here has to redact
/// a tag.
fn union_refusal(
    union: &Union,
    value: &Value,
    path: &[String],
    key: &str,
    scope: Scope<'_>,
) -> Option<TableRefusal> {
    let Some(table) = value.as_object() else {
        return report(scope, path, not_a_table(key, &text_of(value)));
    };
    let tags: Vec<&str> = union.tags().collect();
    let Some(tag) = table.get(&union.tag).and_then(Value::as_str) else {
        return report(
            scope,
            path,
            format!(
                "{key} names no {} of its own; try one of {}",
                union.tag,
                tags.join(", ")
            ),
        );
    };
    let Some(arm) = union.arm(tag) else {
        return report(
            scope,
            path,
            format!(
                "{key} is \"{tag}\", which is not a kind {key} takes; try one of {}",
                tags.join(", ")
            ),
        );
    };
    let tag_key = below(key, &segment(&union.tag));
    for arm_field in &arm.fields {
        let arm_key = below(key, &segment(&arm_field.key));
        let Some(held) = table.get(&arm_field.key) else {
            if arm_field.editable && arm.requires(&arm_field.key) {
                let refused = required_field(&arm_key, &tag_key, &arm.tag);
                if let Some(refused) = report(scope, path, refused.text) {
                    return Some(refused);
                }
            }
            continue;
        };
        if held.is_null() {
            continue;
        }
        let arm_path = below_path(path, &arm_field.key);
        if let Some(refused) = scalar_refusal(arm_field, held, &arm_key, &arm_path, scope) {
            return Some(refused);
        }
    }
    None
}

/// Where `path` leaves `table`: nothing there, the value, or a non-table
/// sitting partway down.
enum Held<'a> {
    /// No such key, or a `null`: unset, and nothing to check.
    Unset,
    /// The value the whole path names.
    Leaf(&'a Value),
    /// A value that is not a table at `path[depth]`, where the schema has
    /// one more table to walk.
    Blocked { value: &'a Value, depth: usize },
}

fn held<'a>(table: &'a Map<String, Value>, path: &[String]) -> Held<'a> {
    let mut nested = Some(table);
    for (depth, name) in path.iter().enumerate() {
        let Some(value) = nested.and_then(|table| table.get(name)) else {
            return Held::Unset;
        };
        if value.is_null() {
            return Held::Unset;
        }
        if depth + 1 == path.len() {
            return Held::Leaf(value);
        }
        match value.as_object() {
            Some(next) => nested = Some(next),
            None => return Held::Blocked { value, depth },
        }
    }
    Held::Unset
}

/// `key` with one more dotted segment. The segment is already quoted the way
/// a row key is, by [`segment`], or is an index.
fn below(key: &str, segment: &str) -> String {
    if key.is_empty() {
        segment.to_owned()
    } else {
        format!("{key}.{segment}")
    }
}

fn below_path(path: &[String], segment: &str) -> Vec<String> {
    let mut out = path.to_vec();
    out.push(segment.to_owned());
    out
}

/// The field with its row key replaced by the full path, so a sentence built
/// by [`super`]'s arms names the key the operator has to fix.
fn named(field: &Field, key: &str) -> Field {
    let mut clone = field.clone();
    clone.key = key.to_owned();
    clone
}

/// What may be printed for `value`: [`crate::dog_settings::REDACTED`] for a
/// secret field, the value itself otherwise.
fn shown_for(secret: bool, value: &Value) -> String {
    shown_value(secret, &text_of(value)).to_owned()
}

/// `value` as the text each arm of [`super`] parses and quotes: bare for a
/// string, compact JSON for anything else.
fn text_of(value: &Value) -> String {
    match value {
        Value::String(text) => text.clone(),
        other => other.to_string(),
    }
}

fn not_a_string(key: &str, shown: &str) -> String {
    format!("{key} is \"{shown}\", which is not a string shep accepts")
}

fn not_a_toggle(key: &str, shown: &str) -> String {
    format!("{key} is \"{shown}\", which is not true or false")
}

fn not_a_list(key: &str, shown: &str) -> String {
    format!("{key} is \"{shown}\", which is not a list shep accepts")
}

fn not_a_table(key: &str, shown: &str) -> String {
    format!("{key} is \"{shown}\", which is not a table shep accepts")
}