Skip to main content

Module reporting

Module reporting 

Source
Expand description

Run reporting: console, NDJSON, JUnit, GitHub and Perfetto sinks. Report sinks: console, NDJSON, JUnit XML, GitHub annotations, Perfetto trace.

The Reporter is the single place test-run output flows through. The runner emits TestEvents; the reporter fans them out to every enabled sink:

  • Console — level-filtered, optionally colorized, ASCII-safe lines so output renders in CI logs and dumb terminals. Failures always show full diagnostics; debug events (LLM calls, selector resolution) are hidden unless -v/-vv is passed.
  • NDJSON (--log-file) — one JSON object per event, each with a ts epoch-millisecond field and a type discriminator. Machine-readable and lossless apart from configured/observed secret redaction: jq '. | select(.type == "step_finished" and .status == "failed")' run.jsonl works out of the box.
  • JUnit XML (--junit) — one <testcase> per test with <failure> entries per failed step, for Jenkins/GitLab/Azure/TeamCity.
  • GitHub — ::error:: workflow commands on failed steps plus a GITHUB_STEP_SUMMARY markdown file, when running in Actions.
  • Perfetto trace (--trace) — a Chrome Trace Event Format file with test/step/LLM-call spans, openable in ui.perfetto.dev.

Structs§

Reporter
Fans run events out to every enabled sink.

Enums§

ColorMode
Color output mode for the console sink.
Level
Console verbosity level. Higher = more output.

Functions§

detect_color
Decides whether the console sink should emit ANSI codes.
print_batch_report
Prints a per-file usage summary followed by the combined cost report for a multi-file (parallel) run, so the overall token/cost totals are explicit.
print_budget_error
Prints a budget exceeded hard error to stderr.
print_budget_warning
Prints a budget exceeded warning to stderr.
print_report
Prints a cost report to stderr after all tests complete.