Skip to content

Commit 5fceeed

Browse files
vashworthsuryaKommana2662
authored andcommitted
Retry iOS golden screenshot on mismatch (flutter#194746)
iOS simulators may sometimes receive system notifications, like this: <img height="500" alt="Image" src="https://github.com/user-attachments/assets/06b01424-7227-4eae-a440-ac6f77f6e0a5" /> This causes the golden to not match. To fix, if the golden doesn't match due to a notification, it will dismiss it and re-do the screenshot and comparison. Fixes flutter#194250. ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [AI contribution guidelines] and understand my responsibilities, or I am not using AI tools. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant in-code documentation (doc comments with `///`). - [x] If this PR introduces a new feature or capability, I created and linked a website documentation issue or PR in [flutter/website] (or verified none is needed). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance. **Note**: The Flutter team is currently trialing the use of [Gemini Code Assist for GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code). Comments from the `gemini-code-assist` bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [AI contribution guidelines]: https://github.com/flutter/flutter/blob/main/CONTRIBUTING.ai.md [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [flutter/website]: https://github.com/flutter/website [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
1 parent 9bae531 commit 5fceeed

1 file changed

Lines changed: 24 additions & 1 deletion

File tree

‎engine/src/flutter/testing/ios_scenario_app/ios/Scenarios/ScenariosUITests/GoldenTestManager.m‎

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,30 @@ - (void)checkGoldenForTest:(XCTestCase*)test rmesThreshold:(double)rmesThreshold
4747
_goldenImage.goldenName);
4848
}
4949

50-
if (![_goldenImage compareGoldenToImage:screenshot.image rmesThreshold:rmesThreshold]) {
50+
BOOL goldenMatches = [_goldenImage compareGoldenToImage:screenshot.image
51+
rmesThreshold:rmesThreshold];
52+
if (!goldenMatches) {
53+
// If the golden does not match, check if there is a system notification. If there is a system
54+
// notification, swipe it away, take a new screenshot, and compare the golden again.
55+
// See: https://github.com/flutter/flutter/issues/194250#issuecomment-6222545727
56+
XCUIApplication* springboard =
57+
[[XCUIApplication alloc] initWithBundleIdentifier:@"com.apple.springboard"];
58+
if (springboard.buttons.count > 0) {
59+
XCUIElement* firstButton = [springboard.buttons elementBoundByIndex:0];
60+
if (firstButton.exists) {
61+
[firstButton swipeUp];
62+
}
63+
[test expectationForPredicate:[NSPredicate predicateWithFormat:@"count == 0"]
64+
evaluatedWithObject:springboard.buttons
65+
handler:nil];
66+
[test waitForExpectationsWithTimeout:30.0 handler:nil];
67+
screenshot = [[XCUIScreen mainScreen] screenshot];
68+
goldenMatches = [_goldenImage compareGoldenToImage:screenshot.image
69+
rmesThreshold:rmesThreshold];
70+
}
71+
}
72+
73+
if (!goldenMatches) {
5174
XCTAttachment* screenshotAttachment = [XCTAttachment attachmentWithImage:screenshot.image];
5275
screenshotAttachment.name = [_goldenImage.goldenName stringByAppendingString:@"_actual.png"];
5376
screenshotAttachment.lifetime = XCTAttachmentLifetimeKeepAlways;

0 commit comments

Comments
 (0)