Skip to content

Fix CI-Tests - solve false-negative caused by poor setup (issue #703) - #702

Merged
hannesa2 merged 1 commit into
masterfrom
show-the-history-view-in-the-wiring-tests
Sep 21, 2026
Merged

hannesa2 merged 1 commit into
masterfrom
show-the-history-view-in-the-wiring-tests

Conversation

@goneng

@goneng goneng commented Sep 20, 2026 •

Copy link
Copy Markdown
Collaborator

These tests expect the window to be showing the history view, while the
screenshot tests leave the application showing the stage view. Which
view the window opens on is a preference that outlives them both, so
once they have run in one job the wiring tests inherit the wrong view
and every later run fails the same way.

The window loads a nib only for the controller it shows, so with the
stage view selected the history nib never loads and commitController
and commitList stay nil. -historyListSelecting: then sets content on
nil and three assertions fail, which is issue #703.

  • Turn PBShowStageView off in -setUp, before the window is built, so
    the suite asks for the view it goes on to assert on.
  • Put the old value back in -tearDown, absent or set, so the suite
    stops leaking the preference to whatever runs after it, following the
    same save and restore PBGitRepositoryFetchArgumentsTests uses.
  • Cross-check the write against [PBGitDefaults showStageView], so
    renaming the key fails loudly here instead of quietly dropping the
    defence.

Test plan:

  • Set PBShowStageView and run PBGitWindowWiringTests: two of the six
    cases fail before this change and all six pass after it. That
    reproduces on macOS 15 with an idle machine, so the failure is not
    about load or about the runner.
  • Rename the key in the test to simulate the drift the cross-check
    guards against: it fails with (([PBGitDefaults showStageView]) is false) failed, so the guard is not vacuous.
  • The full unit suite stays green, 148 tests.
  • The preference still holds its previous value once the suite has run.

CI on this branch cannot demonstrate the fix, because the change touches
only GitXTests, which is not one of the paths the workflow watches for
UI relevance. The baseline app therefore never launches, the preference
is never set, and the run would be green with or without this change.
The local red to green above is the evidence. Landing this and then
re-running a pull request that does touch UI paths - #700 is the
obvious one, since it currently fails these two cases - would show it on
the runner.

Note this is not the same fix as #700, and neither replaces the other.
#700 makes the screenshot tests reach their views by key equivalent
instead of walking the menu bar, which is worth having on its own, but
it still leaves the application on the commit view and so does not clear
this.

Left for a follow-up: the screenshot tests still leak PBShowStageView
into the shared defaults domain, so any other suite that depends on
which view the window opens on stays exposed. This change only defends
the wiring tests.

These tests expect the window to be showing the history view, while the
screenshot tests leave the application showing the stage view, so once
both have run in one job every later run fails the same way.

* turn PBShowStageView off in -setUp, before the window is built, so
  the suite asks for the view it goes on to assert on
* put the old value back in -tearDown, absent or set, so the suite
  stops leaking the preference to whatever runs after it
@goneng
goneng force-pushed the show-the-history-view-in-the-wiring-tests branch from 8176c3f to be838ee Compare September 20, 2026 20:42
@goneng

goneng commented Sep 20, 2026

Copy link
Copy Markdown
Collaborator Author

Raised in review, and worth recording here:

the key is hardcoded as @"PBShowStageView" rather than shared with
PBGitDefaults. That matches PBGitRepositoryFetchArgumentsTests, so
it's house style, but a rename of the constant wouldn't break the test

  • it would silently stop defending.

That is right, and the answer was already in the test this one was
modelled on. PBGitRepositoryFetchArgumentsTests writes the raw key and
then asserts the accessor agrees:

[[NSUserDefaults standardUserDefaults] setInteger:setting forKey:@"PBPruneOnFetch"];
XCTAssertEqual([PBGitDefaults pruneOnFetch], setting);

I had carried over the save and restore but not that cross-check.
Amended to add it:

[[NSUserDefaults standardUserDefaults] setBool:NO forKey:@"PBShowStageView"];
XCTAssertFalse([PBGitDefaults showStageView], @"the preference this suite turns off is read under some other name now, so it is no longer being turned off");

I kept the string rather than exposing a constant from PBGitDefaults,
because that turns a test-only fix into a change to production headers
for no behaviour. The cross-check buys the same protection without it.

Checked that the guard is not vacuous, by renaming the key in the test
to simulate the drift being guarded against:

  • preference set, suite run as it stands: six of six pass
  • key renamed: fails with
    (([PBGitDefaults showStageView]) is false) failed - the preference this suite turns off is read under some other name now

So a rename now fails at once, in every environment, rather than
dropping the defence quietly and coming back later as the same
confusing CI failure.

@goneng goneng changed the title Fix the wiring tests inheriting whichever view ran last Fix CI-Tests - solve false-negative caused by poor setup Sep 20, 2026
@goneng goneng changed the title Fix CI-Tests - solve false-negative caused by poor setup Fix CI-Tests - solve false-negative caused by poor setup (issue #643) Sep 20, 2026
@goneng
goneng marked this pull request as ready for review September 20, 2026 21:06
@goneng

goneng commented Sep 20, 2026

Copy link
Copy Markdown
Collaborator Author

@hannesa2 a note on ordering, since this one blocks the others.

#693, #697 and #700 are all currently red on the same two
PBGitWindowWiringTests cases that this fixes. #686, #692 and #701 are
green, but only because their last runs built their baseline from
722a9022, which predates #695. Rebasing any of them onto current
master turns them red too, so all the open pull requests are
potentially affected and this is worth taking first.

A suggested order, if it helps:

  1. This one.
  2. Rebase Stop the screenshot tests spending a minute changing view #700 on top of it. It fails these exact two cases today, so it
    goes red to green and shows the fix working on the runner. That is
    worth doing on its own before the rest, because this pull request's
    own CI cannot demonstrate it: the change touches only GitXTests,
    which is not one of the paths the workflow watches for UI relevance,
    so the baseline app never launches and the preference is never set.
    The evidence is the local red to green in the description.
  3. Then Support choosing the date-format for the history pane's commit date #693, Cosmetic: Fix the checked-out branch badge being dim and never updating #697 and Fix the history list still claiming to update after it was cleaned up #686 in any order.

#700 is not an alternative to this one. It makes the screenshot tests
reach their views by key equivalent rather than walking the menu bar,
which is a real improvement, but testStagingTabScreenshot still leaves
the application on the commit view, so the preference is still set
afterwards. Both are worth having.

Background is in #703. #643 is a different failure in the same test
class, still open, and #686 covers it.

@goneng goneng changed the title Fix CI-Tests - solve false-negative caused by poor setup (issue #643) Fix CI-Tests - solve false-negative caused by poor setup (issue #703) Sep 20, 2026
@goneng goneng added the pipeline CI pipeline label Sep 20, 2026
@goneng

goneng commented Sep 20, 2026

Copy link
Copy Markdown
Collaborator Author

( @hannesa2 , pls merge this one first - I will rebase all the other pending PRs on top of it afterwards )

@goneng goneng added the test Test related label Sep 20, 2026
@hannesa2

Copy link
Copy Markdown
Contributor

Thanks, then let's start with this one

@hannesa2
hannesa2 merged commit 3ea645c into master Sep 21, 2026
3 checks passed
@hannesa2
hannesa2 deleted the show-the-history-view-in-the-wiring-tests branch September 21, 2026 04:24
@goneng

goneng commented Sep 21, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks, then let's start with this one

Great!
next in line, by order of simplicity/importance:

  1. Fix the history list still claiming to update after it was cleaned up #686 - Solve another CI-tests instability and performance
  2. Stop the screenshot tests spending a minute changing view #700 - Improve UI-CI-tests performance
  3. Cosmetic: Fix the checked-out branch badge being dim and never updating #697 - Cosmetic: Fix the colors of the V-mark of the "currently selected" branch
  4. Support choosing the date-format for the history pane's commit date #693 - UI Enhancement: Support multiple date-formats in the commit-history pane

@goneng goneng added test Test related and removed test Test related pipeline CI pipeline labels Sep 21, 2026
hannesa2 added a commit that referenced this pull request Sep 24, 2026
- Drop the PBGitWindowWiringTests -view workaround: #702 already
  turns off PBShowStageView in -setUp, so -selectCurrentBranch runs
  synchronously and commitController is wired before -setUp returns.
- Assert the Preferences window actually appears in both Settings
  screenshot tests, so they fail fast instead of silently passing
  without a screenshot when the window can not be found.
hannesa2 added a commit that referenced this pull request Sep 24, 2026
- Drop the PBGitWindowWiringTests -view workaround: #702 already
  turns off PBShowStageView in -setUp, so -selectCurrentBranch runs
  synchronously and commitController is wired before -setUp returns.
- Assert navigation to Tree View and the file-view scope bar buttons
  actually succeed, so the four Tree View screenshot tests fail fast
  instead of silently passing without a screenshot when a step can
  not be found.
- Drop leftover debug NSLog/screenshot scaffolding from
  testTreeViewSourceTabScreenshot.
hannesa2 added a commit that referenced this pull request Sep 27, 2026
- Drop the PBGitWindowWiringTests -view workaround: #702 already
  turns off PBShowStageView in -setUp, so -selectCurrentBranch runs
  synchronously and commitController is wired before -setUp returns.
- Assert navigation to Tree View and the file-view scope bar buttons
  actually succeed, so the four Tree View screenshot tests fail fast
  instead of silently passing without a screenshot when a step can
  not be found.
- Drop leftover debug NSLog/screenshot scaffolding from
  testTreeViewSourceTabScreenshot.
hannesa2 added a commit that referenced this pull request Sep 27, 2026
- Drop the PBGitWindowWiringTests -view workaround: #702 already
  turns off PBShowStageView in -setUp, so -selectCurrentBranch runs
  synchronously and commitController is wired before -setUp returns.
- Assert navigation to Tree View and the file-view scope bar buttons
  actually succeed, so the four Tree View screenshot tests fail fast
  instead of silently passing without a screenshot when a step can
  not be found.
- Drop leftover debug NSLog/screenshot scaffolding from
  testTreeViewSourceTabScreenshot.
hannesa2 added a commit that referenced this pull request Sep 30, 2026
- Drop the PBGitWindowWiringTests -view workaround: #702 already
  turns off PBShowStageView in -setUp, so -selectCurrentBranch runs
  synchronously and commitController is wired before -setUp returns.
- Assert navigation to Tree View and the file-view scope bar buttons
  actually succeed, so the four Tree View screenshot tests fail fast
  instead of silently passing without a screenshot when a step can
  not be found.
- Drop leftover debug NSLog/screenshot scaffolding from
  testTreeViewSourceTabScreenshot.
hannesa2 added a commit that referenced this pull request Oct 4, 2026
- Drop the PBGitWindowWiringTests -view workaround: #702 already
  turns off PBShowStageView in -setUp, so -selectCurrentBranch runs
  synchronously and commitController is wired before -setUp returns.
- Assert navigation to Tree View and the file-view scope bar buttons
  actually succeed, so the four Tree View screenshot tests fail fast
  instead of silently passing without a screenshot when a step can
  not be found.
- Drop leftover debug NSLog/screenshot scaffolding from
  testTreeViewSourceTabScreenshot.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test Test related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants