mix quality.verify (ExQuality v0.14.0)

View Source

Runs mix quality and attests that what ran was the full gate.

A green run is not by itself evidence of a green gate. --profile, --test-scope, --quick, --skip and --until-first-failure all produce "status": "ok", and each of them checks less. That is the right design for the inner loop, but the claim made about such a run afterwards is where it goes wrong: when an agent runs the gate unattended and reports "gate green" into a pull request or a commit message, nobody observed the run, and the narrow one and the full one produce the same three words.

This task is the check nobody has to remember to make:

mix quality.verify

It runs the gate - printing exactly what mix quality prints - and then attests over the same results. It exits 0 when the run attests and non-zero when it does not, naming every reason at once:

Not a full gate: run used profile :loop and Dialyzer was skipped (--quick).

What attests

A run attests when all of the following hold:

  • every stage passed ("status": "ok");
  • no profile was used;
  • the test scope was "all" - a :changed scope that fell back to the full suite genuinely ran everything, reports "all", and attests;
  • quick mode was off;
  • no stage was skipped for a reason that names this run (skip_kind "run": --quick, --skip, a profile, --until-first-failure);
  • coverage was measured, when the project measures coverage at all.

A stage skipped for a project-level reason (:sobelow not installed, disabled in .quality.exs) does not fail the attestation. It is named in the output instead, because it is a standing gap in what the project checks at all, and a fuller run cannot close it:

Full gate green: scope all, no profile, 9 stages considered.
Not checked by this project at all: Sobelow (:sobelow not installed)

Those are two different facts, and the reader needs both.

Flags

Takes the same flags as mix quality and passes them through. A flag that narrows the run makes the attestation fail, which is the point: run this the way CI runs the gate, with no flags. --report PATH still writes the caller's report where it asked.

What this does not prove

mix quality.verify attests that the run was not narrowed. It cannot attest that the gate is strong. A project can weaken .quality.exs - drop a stage, add a permissive profile - or lower a coverage threshold, and then attest honestly against the weakened gate. Guarding the gate's own configuration is a different mechanism (a config-change guard against a base ref), and a caller that treats this attestation as proof of it is claiming more than was checked.

Summary

Functions

Runs the gate and attests, failing the task when the run does not attest.

Functions

run(args)

@spec run([String.t()]) :: :ok

Runs the gate and attests, failing the task when the run does not attest.