Fix bug in test_golden_comparator, add an e2e test. #174459
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
the
tool
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request addresses a bug in TestGoldenComparator that caused crashes on golden test failures. The fix involves broadening the exception handling in test_golden_comparator.dart from Exception to Object? to catch any thrown object. Additionally, a new integration test, test_golden_comparator_test.dart, has been added with tests for both successful and failed comparison scenarios to verify the fix. My review includes a suggestion to refactor the new test to reduce code duplication for better maintainability.
packages/flutter_tools/test/integration.shard/test_golden_comparator_test.dart
Outdated
Show resolved
Hide resolved
bkonyi
approved these changes
Aug 26, 2025
| printOnFailure(logger.errorText); | ||
| }); | ||
|
|
||
| // Cannot be in 'setUp' because it implicitly uses [global] state that comes from 'testUsingContext'. |
Contributor
There was a problem hiding this comment.
There's calls to global in this function, so it's hardly implicit 😜
| expect(result, isA<TestGoldenComparisonDone>()); | ||
| }); | ||
|
|
||
| testUsingContext('failed comparison is failed but not crashed', () async { |
Contributor
There was a problem hiding this comment.
Consider leaving a breadcrumb to point to the original issue this regression test was written for.
| print(jsonEncode({'success': success})); | ||
| } on Exception catch (ex) { | ||
| print(jsonEncode({'success': false, 'message': '\$ex'})); | ||
| } on Object? catch (e) { |
Contributor
There was a problem hiding this comment.
Out of curiosity, what's being thrown that wasn't caught by Exception?
matanlurey
added
the
autosubmit
Contributor
|
autosubmit label was removed for flutter/flutter/174459, because - The status or check suite Google testing has failed. Please fix the issues identified (or deflake) before re-applying this label. |
auto-submit
bot
removed
the
autosubmit
matanlurey
added
the
autosubmit
auto-submit
bot
removed
the
autosubmit
Contributor
|
autosubmit label was removed for flutter/flutter/174459, because - The status or check suite Windows plugin_test has failed. Please fix the issues identified (or deflake) before re-applying this label. |
matanlurey
added
the
autosubmit
flutter-dashboard
bot
removed
the
autosubmit
matanlurey
added
cp: beta
flutteractionsbot
pushed a commit
to flutteractionsbot/flutter
that referenced
this pull request
Aug 26, 2025
Closes flutter#174267. The actual fix is 1-line, but of course there is no test suite to verify that, so that took the bulk of the time.
flutteractionsbot
pushed a commit
to flutteractionsbot/flutter
that referenced
this pull request
Aug 26, 2025
Closes flutter#174267. The actual fix is 1-line, but of course there is no test suite to verify that, so that took the bulk of the time.
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 27, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 27, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 27, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 27, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 27, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 27, 2025
matanlurey
added
cp: stable
matanlurey
added a commit
to matanlurey/flutter
that referenced
this pull request
Aug 27, 2025
Closes flutter#174267. The actual fix is 1-line, but of course there is no test suite to verify that, so that took the bulk of the time.
matanlurey
added a commit
to matanlurey/flutter
that referenced
this pull request
Aug 27, 2025
Closes flutter#174267. The actual fix is 1-line, but of course there is no test suite to verify that, so that took the bulk of the time.
auto-submit bot
pushed a commit
to flutter/packages
that referenced
this pull request
Aug 28, 2025
flutter/flutter@c65f01d...da5523a 2025-08-27 [email protected] [native assets] Roll dependencies (flutter/flutter#174522) 2025-08-27 [email protected] Roll Skia from 8cf2faada2b5 to 7e6da45059c5 (5 revisions) (flutter/flutter#174523) 2025-08-27 [email protected] [Android] Restrict AOT shared library engine flag to trusted paths (flutter/flutter#173359) 2025-08-27 [email protected] Roll Fuchsia Linux SDK from L5zGzsIWIS8N36AFQ... to bHYRvLv2Dg56RWZF2... (flutter/flutter#174518) 2025-08-27 [email protected] Roll Packages from 1ef712e to 86fbeec (7 revisions) (flutter/flutter#174521) 2025-08-27 [email protected] Roll Skia from 448a0d0e57e3 to 8cf2faada2b5 (1 revision) (flutter/flutter#174510) 2025-08-27 [email protected] Roll Skia from 4703976a4dae to 448a0d0e57e3 (3 revisions) (flutter/flutter#174494) 2025-08-27 [email protected] Fix SliverMainAxisGroup and SliverCrossAxisGroup gestures' local positions. (flutter/flutter#174265) 2025-08-27 [email protected] Fix lock up when window resized with merged UI and platform threads. (flutter/flutter#172893) 2025-08-27 [email protected] Roll Skia from dc2872de506f to 4703976a4dae (1 revision) (flutter/flutter#174489) 2025-08-27 [email protected] Roll Dart SDK from db45c0ce46f9 to 89023922f96d (2 revisions) (flutter/flutter#174481) 2025-08-27 [email protected] Migrate examples and defaults to `WidgetState` (flutter/flutter#174421) 2025-08-27 [email protected] Roll Skia from 8e48b9e6d67b to dc2872de506f (7 revisions) (flutter/flutter#174479) 2025-08-27 [email protected] SnackBar with action no longer auto-dismiss (flutter/flutter#173084) 2025-08-26 [email protected] Roll Dart SDK from 9054cd8af73c to db45c0ce46f9 (1 revision) (flutter/flutter#174438) 2025-08-26 [email protected] Roll Skia from f941bfab7c09 to 8e48b9e6d67b (4 revisions) (flutter/flutter#174468) 2025-08-26 [email protected] Fix bug in test_golden_comparator, add an e2e test. (flutter/flutter#174459) 2025-08-26 [email protected] fix typo in test_profile/README.md (flutter/flutter#174384) 2025-08-26 [email protected] Remove CP labels on not-merged PRs, and explain why (flutter/flutter#174448) 2025-08-26 [email protected] Increase testing coverage and maintainability of android manifest parsing logic (flutter/flutter#174070) 2025-08-26 [email protected] [web] Add test that pictures are not rasterized when clipped out (flutter/flutter#174452) 2025-08-26 [email protected] Roll Skia from 538260c45b4a to f941bfab7c09 (3 revisions) (flutter/flutter#174450) 2025-08-26 [email protected] fixes the vulkan image layout transitions for mipmap generation (flutter/flutter#173884) 2025-08-26 [email protected] Move flakey iOS tests to bringup (flutter/flutter#174446) 2025-08-26 [email protected] [Impeller] Make sure inline passes always do a clear action. (flutter/flutter#174083) 2025-08-26 [email protected] Roll Skia from 21214d63fc40 to 538260c45b4a (2 revisions) (flutter/flutter#174441) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC [email protected] on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 28, 2025
mboetger
pushed a commit
to mboetger/flutter
that referenced
this pull request
Sep 18, 2025
Closes flutter#174267. The actual fix is 1-line, but of course there is no test suite to verify that, so that took the bulk of the time.
korca0220
pushed a commit
to korca0220/flutter
that referenced
this pull request
Sep 22, 2025
Closes flutter#174267. The actual fix is 1-line, but of course there is no test suite to verify that, so that took the bulk of the time.
Jaineel-Mamtora
pushed a commit
to Jaineel-Mamtora/flutter_forked
that referenced
this pull request
Sep 24, 2025
Closes flutter#174267. The actual fix is 1-line, but of course there is no test suite to verify that, so that took the bulk of the time.
9 tasks
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Nov 12, 2025
lucaantonelli
pushed a commit
to lucaantonelli/flutter
that referenced
this pull request
Nov 21, 2025
Closes flutter#174267. The actual fix is 1-line, but of course there is no test suite to verify that, so that took the bulk of the time.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #174267.
The actual fix is 1-line, but of course there is no test suite to verify that, so that took the bulk of the time.