Expand description
Does it build?
The check that makes coding agents usable. A model that has been told the code compiles will say the code compiles; a compiler that has been run either exits zero or does not. Everything else in this crate is machinery around that difference.
§Which compiler, and why the checker refuses to exist without knowing
“It type-checks” is not a claim about a patch, it is a claim about a patch
and a compiler. The same file passes tsc 5.4 and fails 5.7; mypy
changes what it rejects with every minor release. Evidence that did not say
which tool produced it would be unfalsifiable six months later, which is the
opposite of what evidence is for.
So CompileChecker::probe runs the tool’s version command first and
records the answer in the checker’s own CheckerId, which the trait then
writes into every piece of evidence. A tool that cannot be identified yields
no checker, and therefore no verdicts — which is the right failure, because
a verdict from an unknown compiler would have been worthless anyway.
§The verdict mapping
| What happened | Verdict |
|---|---|
| exit 0 | Pass |
| non-zero exit | Fail, with the diagnostics |
| killed at the timeout | Inconclusive |
| the sandbox refused, or the tool would not start | Inconclusive |
The last two rows are the ones worth stating. A build that timed out has not
shown the code to be broken, so calling it Fail would send a repair loop
chasing a bug that is not there — and calling it Pass would commit code
nobody compiled. Inconclusive is neither, and it does not commit.
§Diagnostics are cut from the front
A failing compiler’s first error is usually the cause and the rest are consequences of it, so what a repair loop needs is the head of the output. That is the opposite of a test runner, which puts its summary at the end, and the two checkers differ accordingly.
Structs§
- Compile
Checker - Compiles or type-checks a workspace, and reports whether it built.
- Compile
Config - A tool to compile or type-check with.
- Sandboxed
Compile - A checker built over a sandbox it does not own.
Enums§
- Probe
Error - The tool could not be identified, so no checker was built.
Constants§
- MAX_
REASON - How much of a compiler’s output reaches the failure reason.
Functions§
- head
- Keep the head of
text, which is where a compiler’s real error is.