Skip to content

Conversation

@zanderso
Copy link
Member

@zanderso zanderso commented Feb 1, 2024

@vashworth
Copy link
Contributor

@zanderso Does flutter_test need to be done too?

Seems there were also some errors in that part https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_arm64%20framework_tests_misc/75/overview

 00:08 +154 ~1: /Volumes/Work/s/w/ir/x/w/flutter/packages/flutter_test/test/widget_tester_test.dart: group retry flag allows test to run multiple times the group with retry flag the test inside it    
 ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
 The following TestFailure was thrown running a test:
 Expected: true
   Actual: <false>
 When the exception was thrown, this was the stack:
 #4      main.<anonymous closure>.<anonymous closure>.<anonymous closure> (file:///Volumes/Work/s/w/ir/x/w/flutter/packages/flutter_test/test/widget_tester_test.dart:56:9)
 #5      testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:183:29)
 <asynchronous suspension>
 #6      TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1019:5)
 <asynchronous suspension>
 <asynchronous suspension>
 (elided one frame from package:stack_trace)
 This was caught by the test expectation on the following line:
   file:///Volumes/Work/s/w/ir/x/w/flutter/packages/flutter_test/test/widget_tester_test.dart line 56
 The test description was:
   the test inside it
 ════════════════════════════════════════════════════════════════════════════════════════════════════
 00:08 +154 ~1 -1: /Volumes/Work/s/w/ir/x/w/flutter/packages/flutter_test/test/widget_tester_test.dart: group retry flag allows test to run multiple times the group with retry flag the test inside it [E]
   Test failed. See exception logs above.
   The test description was: the test inside it
   
 Retry: group retry flag allows test to run multiple times the group with retry flag the test inside it
00:09 +180 ~2: /Volumes/Work/s/w/ir/x/w/flutter/packages/flutter_test/test/widget_tester_test.dart: testWidget retry flag allows test to run multiple times the test with retry flag                   
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following TestFailure was thrown running a test:
Expected: true
  Actual: <false>
When the exception was thrown, this was the stack:
#4      main.<anonymous closure>.<anonymous closure> (file:///Volumes/Work/s/w/ir/x/w/flutter/packages/flutter_test/test/widget_tester_test.dart:65:7)
#5      testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:183:29)
<asynchronous suspension>
#6      TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1019:5)
<asynchronous suspension>
<asynchronous suspension>
(elided one frame from package:stack_trace)
This was caught by the test expectation on the following line:
  file:///Volumes/Work/s/w/ir/x/w/flutter/packages/flutter_test/test/widget_tester_test.dart line 65
The test description was:
  the test with retry flag
════════════════════════════════════════════════════════════════════════════════════════════════════
00:09 +180 ~2 -1: /Volumes/Work/s/w/ir/x/w/flutter/packages/flutter_test/test/widget_tester_test.dart: testWidget retry flag allows test to run multiple times the test with retry flag [E]            
  Test failed. See exception logs above.
  The test description was: the test with retry flag
  
Retry: testWidget retry flag allows test to run multiple times the test with retry flag

Copy link
Contributor

@vashworth vashworth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with question above: #142721 (comment)

@zanderso zanderso force-pushed the no-shuffle-dev-test branch from 14de52d to 3063d2b Compare February 1, 2024 17:25
@github-actions github-actions bot added a: tests "flutter test", flutter_test, or one of our tests framework flutter/packages/flutter repository. See also f: labels. labels Feb 1, 2024
@zanderso
Copy link
Member Author

zanderso commented Feb 1, 2024

@vashworth Thanks for spotting that. I've updated the PR to disable shuffling for gen_defaults tests (hopefully) the right way, and also added the tag to widget_tester_test.dart.

Copy link
Contributor

@vashworth vashworth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still LGTM!

@zanderso zanderso added warning: land on red to fix tree breakage autosubmit Merge PR when tree becomes green via auto submit App labels Feb 1, 2024
Comment on lines 5 to 9
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please revert this. The test isn't failing. (Well, it fails ones and then is retried to verify the retry logic, what you saw in the log is the expected first failure. Overall the test passes with the seed.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@zanderso zanderso removed warning: land on red to fix tree breakage autosubmit Merge PR when tree becomes green via auto submit App labels Feb 1, 2024
Also disable shuffling for widget_tester_test.dart
@zanderso zanderso force-pushed the no-shuffle-dev-test branch from 3063d2b to 9ee0ed3 Compare February 1, 2024 18:07
@github-actions github-actions bot removed a: tests "flutter test", flutter_test, or one of our tests framework flutter/packages/flutter repository. See also f: labels. labels Feb 1, 2024
@zanderso zanderso merged commit 52fdc01 into flutter:master Feb 1, 2024
@zanderso zanderso deleted the no-shuffle-dev-test branch February 1, 2024 18:09
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 2, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 2, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 2, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 2, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 2, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 2, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 3, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 3, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 4, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 4, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 5, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 5, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 5, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 5, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants