dstest 0.1.6

Deterministic Simulation Testing for containerised services
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125

# dst

Fault injection and the oracle verification system. Exposes `dstest.dst.step`,
`dstest.dst.run_steps`, `dstest.dst.clear`, and `dstest.dst.oracle.*`.

## `dstest.dst.step()`

Applies the next fault in the deterministic sequence. Returns a table describing
what happened, or `{ more = false }` when the sequence is exhausted.

```lua
local result = dstest.dst.step()
-- result = { fault = "pause", subject = "docker/abc123", round = 1,
--            total_rounds = 10, remaining = 9, more = true }
```

| Field | Type | Description |
|-------|------|-------------|
| `fault` | string | Fault type applied (`"pause"`, `"kill"`, `"deprive:disk"`, etc.) |
| `subject` | string | Subject ID (`"<substrate>/<id>"`) |
| `round` | number | Current round (1-indexed) |
| `total_rounds` | number | Total rounds in this experiment |
| `remaining` | number | Steps remaining |
| `more` | boolean | Whether more steps are available |
| `oracle` | table | Present only when oracle is enabled (see below) |

In `single` accumulation mode, previous faults are cleared (with `step_delay` ms
wait) before applying the next. In `accumulate` mode, faults stack.

When the oracle is enabled, `step` runs all predicates after applying the fault
and attaches an `oracle` sub-table: `{ passed, total_checks, passed_checks, failed_checks }`.

## `dstest.dst.run_steps(n)`

Runs `n` steps and returns an array of result tables (same shape as `step`).
Stops early if the sequence is exhausted.

```lua
local results = dstest.dst.run_steps(5)
for _, r in ipairs(results) do
    dstest.info("round " .. r.round .. ": " .. r.fault)
end
```

## `dstest.dst.clear(subject)`

Clears all active faults on a subject.

```lua
dstest.dst.clear(subject)
```

## Oracle

Automated verification of system properties during fault injection. Predicates are
checked after each `step`/`run_steps` round; invariants are checked continuously.

### `dstest.dst.oracle.predicate(name, fn)`

Registers a health predicate checked after each fault. The function receives
`(subject, fault, round)` and returns `true` or `{ passed, message? }`.

```lua
dstest.dst.oracle.predicate("http_health", function(subject, fault, round)
    if fault == "pause" or fault == "kill" then return true end
    local ok, resp = pcall(dstest.net.http, subject, "GET", "/health")
    if not ok or resp.status ~= 200 then
        return { false, "health check failed" }
    end
    return true
end)
```

### `dstest.dst.oracle.invariant(name, fn)`

Registers an invariant checked throughout the experiment. Returns `true` or
`{ passed, message? }`.

### `dstest.dst.oracle.run(fn)`

Runs a function with the oracle enabled, returns a report. Automatically enables
checking during `step()`/`run_steps()` and disables it after.

```lua
local report = dstest.dst.oracle.run(function()
    dstest.dst.run_steps(10)
end)

if not report.passed then
    for _, f in ipairs(report.failures) do
        dstest.warn(f.type .. " '" .. f.name .. "': " .. f.error)
    end
    error("oracle verification failed")
end
```

### `dstest.dst.oracle.enable()` / `disable()` / `reset()`

Manual control of oracle checking state. `enable` resets the report. `reset`
clears recorded results without changing the enabled flag.

### `dstest.dst.oracle.report()`

Returns the current report without modifying state.

### Report shape

| Field | Type | Description |
|-------|------|-------------|
| `passed` | boolean | Overall pass/fail |
| `total_checks` | number | Total checks performed |
| `passed_checks` | number | Checks that passed |
| `failed_checks` | number | Checks that failed |
| `failures` | array | List of failure records |

### Failure record

| Field | Type | Description |
|-------|------|-------------|
| `type` | string | `"predicate"` or `"invariant"` |
| `name` | string | Name of the failed check |
| `round` | number | Round number (predicates only) |
| `fault` | string | Fault applied (predicates only) |
| `subject` | string | Subject ID (predicates only) |
| `error` | string | Error message |