Repository navigation
Fix CI-Tests - solve false-negative caused by poor setup (issue #703) - #702
Conversation
These tests expect the window to be showing the history view, while the screenshot tests leave the application showing the stage view, so once both have run in one job every later run fails the same way. * turn PBShowStageView off in -setUp, before the window is built, so the suite asks for the view it goes on to assert on * put the old value back in -tearDown, absent or set, so the suite stops leaking the preference to whatever runs after it
8176c3f to
be838ee
Compare
|
Raised in review, and worth recording here:
That is right, and the answer was already in the test this one was [[NSUserDefaults standardUserDefaults] setInteger:setting forKey:@"PBPruneOnFetch"];
XCTAssertEqual([PBGitDefaults pruneOnFetch], setting);I had carried over the save and restore but not that cross-check. [[NSUserDefaults standardUserDefaults] setBool:NO forKey:@"PBShowStageView"];
XCTAssertFalse([PBGitDefaults showStageView], @"the preference this suite turns off is read under some other name now, so it is no longer being turned off");I kept the string rather than exposing a constant from Checked that the guard is not vacuous, by renaming the key in the test
So a rename now fails at once, in every environment, rather than |
|
@hannesa2 a note on ordering, since this one blocks the others. #693, #697 and #700 are all currently red on the same two A suggested order, if it helps:
#700 is not an alternative to this one. It makes the screenshot tests Background is in #703. #643 is a different failure in the same test |
|
( @hannesa2 , pls merge this one first - I will rebase all the other pending PRs on top of it afterwards ) |
|
Thanks, then let's start with this one |
Great!
|
- Drop the PBGitWindowWiringTests -view workaround: #702 already turns off PBShowStageView in -setUp, so -selectCurrentBranch runs synchronously and commitController is wired before -setUp returns. - Assert the Preferences window actually appears in both Settings screenshot tests, so they fail fast instead of silently passing without a screenshot when the window can not be found.
- Drop the PBGitWindowWiringTests -view workaround: #702 already turns off PBShowStageView in -setUp, so -selectCurrentBranch runs synchronously and commitController is wired before -setUp returns. - Assert navigation to Tree View and the file-view scope bar buttons actually succeed, so the four Tree View screenshot tests fail fast instead of silently passing without a screenshot when a step can not be found. - Drop leftover debug NSLog/screenshot scaffolding from testTreeViewSourceTabScreenshot.
- Drop the PBGitWindowWiringTests -view workaround: #702 already turns off PBShowStageView in -setUp, so -selectCurrentBranch runs synchronously and commitController is wired before -setUp returns. - Assert navigation to Tree View and the file-view scope bar buttons actually succeed, so the four Tree View screenshot tests fail fast instead of silently passing without a screenshot when a step can not be found. - Drop leftover debug NSLog/screenshot scaffolding from testTreeViewSourceTabScreenshot.
- Drop the PBGitWindowWiringTests -view workaround: #702 already turns off PBShowStageView in -setUp, so -selectCurrentBranch runs synchronously and commitController is wired before -setUp returns. - Assert navigation to Tree View and the file-view scope bar buttons actually succeed, so the four Tree View screenshot tests fail fast instead of silently passing without a screenshot when a step can not be found. - Drop leftover debug NSLog/screenshot scaffolding from testTreeViewSourceTabScreenshot.
- Drop the PBGitWindowWiringTests -view workaround: #702 already turns off PBShowStageView in -setUp, so -selectCurrentBranch runs synchronously and commitController is wired before -setUp returns. - Assert navigation to Tree View and the file-view scope bar buttons actually succeed, so the four Tree View screenshot tests fail fast instead of silently passing without a screenshot when a step can not be found. - Drop leftover debug NSLog/screenshot scaffolding from testTreeViewSourceTabScreenshot.
- Drop the PBGitWindowWiringTests -view workaround: #702 already turns off PBShowStageView in -setUp, so -selectCurrentBranch runs synchronously and commitController is wired before -setUp returns. - Assert navigation to Tree View and the file-view scope bar buttons actually succeed, so the four Tree View screenshot tests fail fast instead of silently passing without a screenshot when a step can not be found. - Drop leftover debug NSLog/screenshot scaffolding from testTreeViewSourceTabScreenshot.
These tests expect the window to be showing the history view, while the
screenshot tests leave the application showing the stage view. Which
view the window opens on is a preference that outlives them both, so
once they have run in one job the wiring tests inherit the wrong view
and every later run fails the same way.
The window loads a nib only for the controller it shows, so with the
stage view selected the history nib never loads and
commitControllerand
commitListstay nil.-historyListSelecting:then sets content onnil and three assertions fail, which is issue #703.
PBShowStageViewoff in-setUp, before the window is built, sothe suite asks for the view it goes on to assert on.
-tearDown, absent or set, so the suitestops leaking the preference to whatever runs after it, following the
same save and restore
PBGitRepositoryFetchArgumentsTestsuses.[PBGitDefaults showStageView], sorenaming the key fails loudly here instead of quietly dropping the
defence.
Test plan:
PBShowStageViewand runPBGitWindowWiringTests: two of the sixcases fail before this change and all six pass after it. That
reproduces on macOS 15 with an idle machine, so the failure is not
about load or about the runner.
guards against: it fails with
(([PBGitDefaults showStageView]) is false) failed, so the guard is not vacuous.CI on this branch cannot demonstrate the fix, because the change touches
only
GitXTests, which is not one of the paths the workflow watches forUI relevance. The baseline app therefore never launches, the preference
is never set, and the run would be green with or without this change.
The local red to green above is the evidence. Landing this and then
re-running a pull request that does touch UI paths - #700 is the
obvious one, since it currently fails these two cases - would show it on
the runner.
Note this is not the same fix as #700, and neither replaces the other.
#700 makes the screenshot tests reach their views by key equivalent
instead of walking the menu bar, which is worth having on its own, but
it still leaves the application on the commit view and so does not clear
this.
Left for a follow-up: the screenshot tests still leak
PBShowStageViewinto the shared defaults domain, so any other suite that depends on
which view the window opens on stays exposed. This change only defends
the wiring tests.