You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
CI: the unit test suite runs twice on every pull request #705
GitXTests is run twice per pull request, in two different steps, and
the second run is incidental rather than intended.
Run unit tests runs make unit-test, which is -only-testing:GitXTests.
Run tests runs make all-tests, which tests every target in the
scheme - so it runs GitXTests again on its way to GitXUITests.
The workflow already notes this, in a TODO above the second step:
Consider running make ui-test here rather than make all-tests. The
scheme tests every target, so this repeats the GitXTests run of the
"Run unit tests" step above; ui-test narrows it to GitXUITests,
which is all the screenshots need.
What it actually costs
Less than it looks, and worth being honest about: the build in the
second step is needed for the UI tests anyway, so the duplicate is the
test execution alone, about 14 seconds on a recent run. Narrowing the
step to make ui-test would save that, not minutes.
The better reason to fix it
The two runs are not equivalent, and that has already caused confusion.
The Run unit tests step happens before Build baseline and run its UI tests; the Run tests step happens after it. Since launching the
application leaves state in the shared defaults domain, the same suite
can pass in the first step and fail in the second, on one build, in one
job. That is precisely how #703 presented, and the duplicate run is what
made it look like a flake rather than a dependency on what ran before.
Two identical Executed 148 tests summaries in one log, which can
disagree, is also just hard to read.
Narrowing Run tests to make ui-test removes the ambiguity. It would
not have fixed #703 - #702 did that, by making the suite independent of
the preference - but it would have made it much easier to see.
Activity