# `ExQuality.Stages.Test`
[🔗](https://github.com/riddler/ex_quality/blob/v0.16.0/lib/ex_quality/stages/test.ex#L1)

Runs the test suite with optional coverage analysis.

- Uses `mix coveralls` if `:excoveralls` is in deps
- Uses `mix test --cover` if it is not, and the project asks for coverage
- Uses `mix test` otherwise, and always in quick mode
- Parses test count, pass/fail counts
- Parses coverage percentage if available
- Reads the coverage threshold from the tool's own config (single source of
  truth), whichever of the two tools is in use

In quick mode: runs `mix test`
(tests must pass, but coverage threshold is not enforced).

## Native coverage

Elixir has shipped coverage since 1.11, so a project without `:excoveralls`
is not a project without coverage. `mix test --cover` is used for those,
reading `test_coverage: [summary: [threshold: N]]` from `mix.exs` the same way
the built-in tool does.

It is used only when the project sets that threshold, or asks for coverage
explicitly with `test: [coverage: true]` in `.quality.exs`. Elixir's built-in
threshold defaults to 90% whether or not a project has ever thought about
coverage, and turning a green run red over a number nobody chose is not a
gate anyone asked for. A stated threshold is the project saying it wants one.

## Umbrellas

An umbrella `mix test` prints one summary line per app, so every summary is
read and summed rather than only the first. Reporting the first app's numbers
as the suite's is worse than reporting none: a confident, specific, false
count ("3 of 12 failed" when the truth is "3 of 4,180 failed") reads as more
trustworthy than no count at all.

Apps with failures are named in the summary. Apps that passed are not, since
the failing ones are what a reader acts on.

Native coverage in an umbrella is measured in two commands, because a per-app
`mix test --cover` only sees its own app's modules: a module exercised by
another app's tests reads as 0%. So the run exports instead
(`--export-coverage`), and `mix test.coverage` aggregates the exports into one
table and one total.

## Findings

A failing test is the most common thing a run has to hand off, so each one is
parsed into a finding carrying the file, the line the test is defined at, the
app, the test module and the assertion message. The alternative is 30 KB of
run log with one failure buried in it.

The failure block's own `stacktrace:` lines are relative to the app, while
the header line under the test's name is relative to the umbrella root, so
the header line is what is read: a path that does not open from where the
report was generated cannot be routed anywhere.

The parse is used only when it accounts for every failure the suite counted.
Three findings for four failures would read as the complete list, which is
worse than showing the log; a short parse falls back to `output` in full.

A failing coverage check reports the modules under the threshold, not the
whole table. A 400-module umbrella prints 400 rows; the handful below the line
are the ones a reader acts on. Modules are reported only when the run actually
failed on coverage, because the threshold applies to the total: a passing run
can hold a low module and there is nothing to do about it.

## Scoped runs

`test: [scope: :changed]`, or `--test-scope changed`, runs only the test files
covering the code that changed. This is the whole reason an agent can afford to
run checks between edits: the suite is most of a run's wall clock, and a
one-file change needs a handful of files rather than all of them. See
`ExQuality.Scope`, which owns resolving the scope and the rule that an empty
resolution runs the full suite rather than reporting a green run of nothing.

Coverage in a scoped run is not lower, it is **absent**. It is reported as
`skipped` with a reason and never as a number, because a percentage measured
over a subset of the suite is not a smaller truth, it is a different and
misleading one, and anything downstream that ratchets a recorded figure would
lower it against a run that never measured.

The result carries what the run actually covered under `meta`, so a
`"status": "ok"` in the report can be interpreted rather than assumed to be a
full green.

# `measures_coverage?`

```elixir
@spec measures_coverage?(keyword()) :: boolean()
```

Whether a full, unnarrowed run under `config` would measure coverage.

`mix quality.verify` asks this to tell "coverage was narrowed away by this
run" from "this project never measures coverage" - the same run-vs-project
distinction `skip_kind` carries for stages, answered here for a number that
is not a stage of its own. Run-level narrowings (`--quick`, a scope) are
ignored; the project-level `test: [coverage: false]` and the absence of any
coverage tool or threshold are not.

# `run`

```elixir
@spec run(keyword()) :: ExQuality.Stage.result()
```

Runs the test stage.

## Config options

- `quick` - Use `mix test` with no coverage at all (default: false)
- `test.args` - Extra arguments for the test command (default: `[]`)
- `test.coverage` - `:auto` (use the project's config) | `true` (always
  measure) | `false` (never measure) (default: `:auto`)
- `test.scope` - `:all` | `:changed` | a glob string (default: `:all`)
- `test.base_ref` - What `:changed` is measured against (default: the
  repository's default branch)

---

*Consult [api-reference.md](api-reference.md) for complete listing*