Skip to content

Enable goldenFileComparator fix for on-device integration tests. #163157

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
auto-submit merged 2 commits into from
Feb 13, 2025
Merged

Enable goldenFileComparator fix for on-device integration tests. #163157

auto-submit merged 2 commits into from
Feb 13, 2025

Conversation

@matanlurey
Copy link
Contributor

@matanlurey matanlurey commented Feb 12, 2025

Closes #143299.
Closes #160043.

A website update is work-in-progress here to add more context for users: flutter/website#11716.

@github-actions github-actions bot added a: tests "flutter test", flutter_test, or one of our tests framework flutter/packages/flutter repository. See also f: labels. f: integration_test The flutter/packages/integration_test plugin labels Feb 12, 2025
jonahwilliams
jonahwilliams approved these changes Feb 12, 2025
View reviewed changes
Copy link
Contributor

@jonahwilliams jonahwilliams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@matanlurey
Copy link
Contributor Author

matanlurey commented Feb 12, 2025

@Piinks Added you as a reviewer more as an FYI since this is a quasi-breaking change.

@matanlurey matanlurey added the autosubmit Merge PR when tree becomes green via auto submit App label Feb 12, 2025
@auto-submit
Copy link
Contributor

auto-submit bot commented Feb 13, 2025

autosubmit label was removed for flutter/flutter/163157, because - The status or check suite Google testing has failed. Please fix the issues identified (or deflake) before re-applying this label.

@auto-submit auto-submit bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Feb 13, 2025
@matanlurey
Copy link
Contributor Author

matanlurey commented Feb 13, 2025

This is blocked by google3, which I believe has its own custom implementation that is now being overwritten.

Investigating.

@matanlurey
Copy link
Contributor Author

matanlurey commented Feb 13, 2025

This is correctly implemented externally; note that:

flutter/packages/flutter_tools/lib/src/test/flutter_platform.dart

Lines 206 to 210 in c4d53f1

if (integrationTest) {
buffer.write('''
IntegrationTestWidgetsFlutterBinding.ensureInitialized();
''');
}

Binding.ensureInitialized() happens quite early:

flutter/packages/flutter_tools/lib/src/test/flutter_platform.dart

Line 250 in c4d53f1

return () => test_config.testExecutable(_testMain);

... and if a test_config is specified, it takes precedence.

For example:

flutter/examples/api/test/flutter_test_config.dart

Lines 9 to 12 in c4d53f1

Future<void> testExecutable(FutureOr<void> Function() testMain) {
// Enable golden file testing using Skia Gold.
return flutter_goldens.testExecutable(testMain, namePrefix: 'api');
}

... which uses:

flutter/packages/flutter_goldens/lib/flutter_goldens.dart

Line 111 in c4d53f1

await testMain();

Note again that (this is confusing) testMain() is called last, after the comparator is implicitly set by Binding.ensureInitialized(), so it's kosher.

@github-actions github-actions bot added the tool Affects the "flutter" command-line tool. See also t: labels. label Feb 13, 2025
jonahwilliams
jonahwilliams approved these changes Feb 13, 2025
View reviewed changes
Copy link
Contributor

@jonahwilliams jonahwilliams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@matanlurey matanlurey added the autosubmit Merge PR when tree becomes green via auto submit App label Feb 13, 2025
@auto-submit auto-submit bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Feb 13, 2025
@auto-submit
Copy link
Contributor

auto-submit bot commented Feb 13, 2025

autosubmit label was removed for flutter/flutter/163157, because - The status or check suite Google testing has failed. Please fix the issues identified (or deflake) before re-applying this label.

@matanlurey matanlurey added the autosubmit Merge PR when tree becomes green via auto submit App label Feb 13, 2025
@auto-submit auto-submit bot added this pull request to the merge queue Feb 13, 2025
Merged via the queue into flutter:master with commit 4ff2dea Feb 13, 2025
143 checks passed
@flutter-dashboard flutter-dashboard bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Feb 13, 2025
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 13, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 13, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
6e309ef 
…ests. (flutter/flutter#163157)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 13, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 13, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
ef8df8c 
…ests. (flutter/flutter#163157)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 14, 2025
Closed
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 14, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 14, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
e0c9186 
…ests. (flutter/flutter#163157)
@Piinks Piinks mentioned this pull request Feb 14, 2025
Open
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 15, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 15, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
db500fe 
…ests. (flutter/flutter#163157)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 15, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 15, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
d94a5e9 
…ests. (flutter/flutter#163157)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 16, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 16, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
d5516f7 
…ests. (flutter/flutter#163157)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 16, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 16, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
a967d0c 
…ests. (flutter/flutter#163157)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 17, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 17, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
74dcec1 
…ests. (flutter/flutter#163157)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 17, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 17, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
4411020 
…ests. (flutter/flutter#163157)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Feb 17, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 17, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
734f228 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 18, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
ca9f8ef 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 18, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
a3002c6 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 18, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
6ef152a 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 19, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
0e75b38 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 19, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
7c4205b 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 19, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
564570a 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 20, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
217e702 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 20, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
f9d0988 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 20, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
681d0e4 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 20, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
d00dd19 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
2c048cf 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
cafd96e 
…ests. (flutter/flutter#163157)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 21, 2025
@engine-flutter-autoroll
4ff2dea Enable goldenFileComparator fix for on-device integration t…
08b4de3 
…ests. (flutter/flutter#163157)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Piinks Piinks Awaiting requested review from Piinks

1 more reviewer

@jonahwilliams jonahwilliams jonahwilliams approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

a: tests "flutter test", flutter_test, or one of our tests f: integration_test The flutter/packages/integration_test plugin framework flutter/packages/flutter repository. See also f: labels. tool Affects the "flutter" command-line tool. See also t: labels.

Projects

None yet

Milestone

No milestone

2 participants

@matanlurey @jonahwilliams