Repository navigation
Description
The post-submit test builder Linux web_tool_tests had a flaky ratio 2.44% for the past 15 days, which is above our 2.00% threshold.
One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_tool_tests/4048,%204047
Commit: a4b5123
Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_tool_tests/4048
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_tool_tests/4047
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_tool_tests/3984
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_tool_tests/3980
Succeeded builds (3 most recent):
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_tool_tests/4075
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_tool_tests/4074
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_tool_tests/4073
Please follow https://github.com/flutter/flutter/wiki/Reducing-Test-Flakiness#fixing-flaky-tests to fix the flakiness and enable the test back after validating the fix (internal dashboard to validate: go/flutter_test_flakiness).
Activity
zanderso commentedon Nov 8, 2021
The flaky error is the following:
zanderso commentedon Nov 9, 2021
@sigmundch Sorry if my information is stale, but is @annagrin still the right person to look into
dwdsissues that come up in the flutter tool?sigmundch commentedon Nov 10, 2021
👍 @annagrin is the right contact. FYI she is out a few days so this will likely not be acted on immediately. Can you clarify on the P1 prioritization, just want to align on expectations. It's my understanding that flutter-web-tools flakiness issues have been an ongoing battle and not something I expect to see it fixed so quickly.
zanderso commentedon Nov 10, 2021
@sigmundch Flutter's flaky test process (go/reduce_flutter_flakiness) treats tests that flake at a rate > 2% as P1 issues. This is currently the only open issue related to flutter-web-tools.
zanderso commentedon Nov 10, 2021
@yjbanov I believe @annagrin won't be able to take a look at this until next week. Do you know of anyone else who might be familiar with this area?
annagrin commentedon Nov 18, 2021
I see two types of errors in the logs:
failure to start dwds's extension backend
stack trace
Opened issue: dart-lang/webdev#1450
failure to connect to chrome
stack trace
Opened issue: #93889
4 remaining items
fluttergithubbot commentedon Dec 13, 2021
Current flaky ratio for the past (up to) 100 commits is 1.00%.
One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_tool_tests/4272
Commit: 16ec56e
Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_tool_tests/4272
Recent test runs:
https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20web_tool_tests
annagrin commentedon Dec 14, 2021
Changed priority to P3 since the flakiness is now below 2%.
fluttergithubbot commentedon Dec 20, 2021
Current flaky ratio for the past (up to) 100 commits is 0.00%. Flaky number: 0; total number: 100.
fluttergithubbot commentedon Dec 27, 2021
Current flaky ratio for the past (up to) 100 commits is 1.00%. Flaky number: 1; total number: 100.
One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_tool_tests/4463
Commit: c584a5e
Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_tool_tests/4463
Recent test runs:
https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20web_tool_tests
zanderso commentedon Dec 28, 2021
Build 4463 looks like the tool gets an unexpected response when trying to connect to the chrome debug port:
fluttergithubbot commentedon Jan 3, 2022
Current flaky ratio for the past (up to) 100 commits is 1.00%. Flaky number: 1; total number: 100.
One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_tool_tests/4463
Commit: c584a5e
Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_tool_tests/4463
Recent test runs:
https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20web_tool_tests
sigmundch commentedon Jan 6, 2022
Indeed - looks like build 4463 is the same as #93256 (comment) (tracked also in #94886)
fluttergithubbot commentedon Jan 10, 2022
Current flaky ratio for the past (up to) 100 commits is 0.00%. Flaky number: 0; total number: 100.
annagrin commentedon Jan 10, 2022
The remaining issue is tracked in #94886. The flakiness does not appear to be critical at this point.
zanderso commentedon Jan 10, 2022
Closing based on comment in #93256 (comment)
github-actions commentedon Jan 24, 2022
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of
flutter doctor -vand a minimal reproduction of the issue.