Repository navigation
Description
The tests expect the application to be showing the history view. The
screenshot tests, which run earlier in the same CI job, leave it showing
the stage view. Which view the window opens on is a preference that
outlives them both, so the wiring tests inherit the wrong one, and from
that point every rerun fails the same way.
Put briefly: #579 built the trap, #695 sprang it.
- Fix the history list selection fighting the sidebar branch #579 added
PBGitWindowWiringTests, inaadd34e4. The suite has
depended on whichever view that preference names since the day it was
written, but nothing ever set it, so it never fired. - Fix the screenshot tests all photographing the same view #695 added the view switching to the screenshot tests, in
fe42261d
(for Screenshot tests capture the same view three times #676), merged asb54bb4b3. From that commit on, the screenshot
tests leave the application on the commit view, and the preference is
persisted.
So the failure is a property of what ran before these tests, not of the
machine's load and not of the branch under test.
Split out of #643, which was filed for a different failure in the same
test class. See the note there.
The wiring
The window shows one content controller at a time, and
-changeContentController: loads a nib only for the controller it is
about to show, by asking that controller for its view. Which one is
shown is decided in PBGitSidebarController:
if ([PBGitDefaults showStageView])
[self selectStage];
else
[self selectCurrentBranch];PBShowStageView is persisted in the app's defaults domain, and
PBGitSidebarController writes it whenever the Stage row is selected.
So when a GitX has been left on the stage view, these tests build a
window whose history nib never loads. commitController and
commitList stay nil, -historyListSelecting: sets content on nil,
and three assertions fail:
PBGitWindowWiringTests.m: ((history.singleCommitSelected) is true) failed
-[testTheFocusedHistoryListNamesTheBranchOnItsSelectedCommit]
-[testACommitCarryingTwoBranchesNamesNeither]
PBGitWindowWiringTests.m: ("(null)") is not equal to ("refs/heads/branch_one")
A diagnostic build reports the state as history: set, commitController: nil, content: 0, selectionIndexes: (null). An array controller answers
an empty index set rather than (null), so the outlet is nil rather
than the selection being empty.
How CI sets it
The screenshot tests drive the real application and switch to
View > Commit. Test methods run alphabetically, so
testStagingTabScreenshot runs last of the four and never switches
back, leaving the app on the stage view and the preference set in the
defaults domain shared by every GitX on the runner. The workflow then
launches that app in Build baseline and run its UI tests, and the
later Run tests step re-runs GitXTests in the same job, where they
inherit it.
Evidence
One CI job, one build, one variable at a time:
| condition | result |
|---|---|
Run unit tests, before the baseline app launches |
148/148 pass |
the same isolated make unit-test, after it launches |
exit=2 |
| whole defaults domain wiped first | exit=0 |
only NSWindow Frame GitX removed |
exit=2 |
only PBShowStageView turned off |
exit=0 |
It reproduces on an idle machine, macOS 15 included, with no CI involved:
defaults write net.phere.GitX PBShowStageView -bool YES
make unit-test ARCH=arm64Two of the six cases fail and the other four pass. The failing cases
take well under a second, which is what rules out load: a load problem
shows up in the 30 second waits, not in these assertions.
Two details that disguised it
- Only the arm64 leg runs any tests at all - every test step is gated on
matrix.abi == 'arm64'- so the intel leg cannot go red and its green
says nothing. That is the whole "always arm64" pattern. - The baseline screenshot cache is only saved by a job that succeeds, so
a failing pull request rebuilds the baseline on every run and keeps
failing, while one green run makes later runs skip the baseline
entirely and pass for the wrong reason.
Fix
#702 makes the suite say which view it needs rather than inherit it:
save PBShowStageView in -setUp, turn it off before the window is
built, and put the old value back in -tearDown.
Still open after that: the screenshot tests continue to leak the
preference into the shared domain, so any other suite that depends on
which view the window opens on stays exposed. Fixing that at source
belongs with the screenshot tests, alongside #700.
Activity
goneng commentedon Sep 20, 2026
How this was found, and what it overturns. Recorded here because three
things that were said about it along the way were wrong, and the wrong
versions are more memorable than the right one.
Corrections
failure was assumed to be the same thing for days. It is not: the
failing cases take well under a second, and it reproduces on an idle
machine. A load problem shows up in the 30 second waits instead.
gated on
matrix.abi == 'arm64', so the intel leg runs no tests atall and its green says nothing. The "always arm64" pattern is that
gate. It was also blamed on macOS 26 for a while, until it reproduced
on macOS 15.
date column formatter. It did not: the formatter is present in runs
that pass once the preference is cleared, and the isolated suite
passes with it earlier in the very same job. Support choosing the date-format for the history pane's commit date #693 was only implicated
because it touches a xib, which makes the "can this PR affect the UI"
gate true and so runs the baseline machinery at all.
Two earlier comments on #643 proposed mechanisms that are also wrong: a
late
NSUserDefaultsDidChangeNotification, and-historyListSelecting:racing a KVO hop on the array controller's selection. Both were deleted
rather than left to mislead. The second is wrong at the root - the
selection is never reached, because the array controller is nil. The
KVO claim was modelled separately across seven array controller
configurations and the selection lands synchronously in all of them.
What settled it
Narrowing one variable at a time inside a single CI job, rather than
comparing runs. The same isolated
make unit-testcommand passes earlyin the job and fails later in it, which rules out the build, the
machine and the branch in one step and leaves only what happened in
between. From there, wiping the defaults domain fixed it, removing the
saved window frame did not, and turning off
PBShowStageViewalone did.Worth carrying for any future CI probe here: the baseline screenshot
cache is only saved by a job that succeeds. A failing pull request
rebuilds the baseline every run and keeps failing, while one green run
makes later runs skip the baseline and pass for the wrong reason. That
happened mid-investigation and produced a false green.
Relationship to #700
#700 reaches the same two commits - #695 as the trigger - by a different
route, and treats the cause as step duration. It replaces the menu bar
traversal with key equivalents, which is worth having on its own, but
testStagingTabScreenshotstill switches to the commit view and neverswitches back, so the preference is still left set. Its own CI run fails
the same three assertions in the same two cases. The two changes are
complementary; neither replaces the other.
Merge pull request #702 from gitx/show-the-history-view-in-the-wiring…
goneng commentedon Sep 21, 2026
Fixed by #702, merged as be838ee: the suite turns PBShowStageView off in -setUp and puts the old value back in -tearDown, so it no longer inherits whichever view ran before it.
That PR names this issue in its title but without a closing keyword, so the issue stayed open after it merged.