Expand description
Acceptance strength: how well a task’s acceptance set tells right from wrong.
A red → green acceptance test shows the task’s work made it pass; it does
not show the test would have failed a wrong implementation. After a
verified verdict, wrong candidates — mutants of the changed code and
model-written near misses — are run against the set: the share it rejects
is its strength. Spec-permitted alternatives are run too: one the set
rejects is a test that encodes a choice the spec did not make, and only a
person can rule on it.
These are the rules; running candidates is the runtime’s.
Structs§
- Acceptance
Strength - The acceptance set’s strength, as recorded with a verdict.
- Counts
- Rejected of judged, for one source.
- Mutants
- Mutants of the changed code, run against the acceptance command only.
- Near
Miss - A model-written near miss: plausible, and wrong about one scenario.
- Permitted
- An alternative the author says the spec allows.
- Ruling
- A person’s ruling that an over-strict finding was wrong: the tests were right, the alternative is not what the spec allows.
- Strength
- One set’s strength.
Enums§
- Gate
- The
min_strengthgate. - Outcome
- What a candidate run showed.
- Permitted
Outcome - What a spec-permitted alternative showed.
- Set
- Which set.
- State
- How far a measurement got.
Functions§
- gate
- The gate for
min. Only a full measurement can meet it: no evidence is not good evidence, and a partial one never satisfies it. - over_
strict - The alternatives the set rejected on an assertion and nobody ruled on: probably over-strict tests.
- strength
- One set’s strength from what ran.
partial: the budget ran out. - summary
0.83 mutants=9/11 near-miss=1/2 unviable=2, ornot measured (…).- trailers
- The trailers of a verified tick:
Acceptance-Strength:(the combined set) and, with held-out tests,Acceptance-Strength-Visible:. - unmeasured
- A strength that measured nothing, with its state.