Repository navigation
Retry iOS golden screenshot on mismatch - #194746
Merged
Merged
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces a 30-second sleep and retry mechanism in GoldenTestManager.m when a golden image comparison fails, aiming to mitigate transient system notifications. Feedback suggests replacing this unconditional 30-second delay with a polling loop that retries comparison every 2 seconds up to 5 times, reducing the performance penalty during actual test failures.
jmagman
reviewed
Oct 19, 2026
|
|
||
| if (![_goldenImage compareGoldenToImage:screenshot.image rmesThreshold:rmesThreshold]) { | ||
| // If the golden does not match. Wait 30 seconds and try again with a new screenshot. | ||
| // Occasionally the first screenshot may include a system notification. See |
Member
There was a problem hiding this comment.
Instead of this, is there a way to do something similar to the devicelab doctor where on test suite setup a XCUIApplication is created with springboard's process id, and then dismissed if there are any showing?
https://github.com/flutter/cocoon/blob/a297e2db6d05dc6b04f7f02960f6f5b9efb72ff8/cipd_packages/device_doctor/tool/infra-dialog/infra-dialogUITests/infra_dialogUITests.swift#L19
vashworth
added
the
autosubmit
auto-submit
Bot
removed
the
autosubmit
Contributor
|
autosubmit label was removed for flutter/flutter/194746, because - The status or check suite Presubmit has failed. Please fix the issues identified (or deflake) before re-applying this label. |
jmagman
approved these changes
Oct 20, 2026
vashworth
added
the
autosubmit
auto-submit
Bot
removed
the
autosubmit
Contributor
|
autosubmit label was removed for flutter/flutter/194746, because - The status or check suite Presubmit has failed. Please fix the issues identified (or deflake) before re-applying this label. |
vashworth
added
the
autosubmit
flutter-dashboard
Bot
removed
the
autosubmit
This was referenced Oct 23, 2026
This was referenced Oct 25, 2026
suryaKommana2662
pushed a commit
to suryaKommana2662/flutter
that referenced
this pull request
Oct 29, 2026
iOS simulators may sometimes receive system notifications, like this: <img height="500" alt="Image" src="https://github.com/user-attachments/assets/06b01424-7227-4eae-a440-ac6f77f6e0a5" /> This causes the golden to not match. To fix, if the golden doesn't match due to a notification, it will dismiss it and re-do the screenshot and comparison. Fixes flutter#194250. ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [AI contribution guidelines] and understand my responsibilities, or I am not using AI tools. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant in-code documentation (doc comments with `///`). - [x] If this PR introduces a new feature or capability, I created and linked a website documentation issue or PR in [flutter/website] (or verified none is needed). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance. **Note**: The Flutter team is currently trialing the use of [Gemini Code Assist for GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code). Comments from the `gemini-code-assist` bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [AI contribution guidelines]: https://github.com/flutter/flutter/blob/main/CONTRIBUTING.ai.md [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [flutter/website]: https://github.com/flutter/website [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
iOS simulators may sometimes receive system notifications, like this:
This causes the golden to not match. To fix, if the golden doesn't match due to a notification, it will dismiss it and re-do the screenshot and comparison.
Fixes #194250.
Pre-launch Checklist
///).If you need help, consider asking for advice on the #hackers-new channel on Discord.
If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance.
Note: The Flutter team is currently trialing the use of Gemini Code Assist for GitHub. Comments from the
gemini-code-assistbot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.