Skip to content

Fix flutter update-packages regression by fixing parameters in "pub get" runner - #116687

Merged
auto-submit[bot] merged 15 commits into
flutter:masterfrom
nehalvpatel:fix-flutter-pub-get-parameters
Jan 6, 2023
Merged

auto-submit[bot] merged 15 commits into
flutter:masterfrom
nehalvpatel:fix-flutter-pub-get-parameters

Conversation

@nehalvpatel

@nehalvpatel nehalvpatel commented Dec 7, 2022 •

Copy link
Copy Markdown
Contributor

With the update to the "pub get" runner in #115801, there were a couple parameters (printProgress and retry) that were left out. This causes flutter update-packages to print unexpected output as seen in #116627.

This PR fixes that by hooking up the parameters.

Pre-launch Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I read the Tree Hygiene wiki page, which explains my responsibilities.
  • I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
  • I signed the CLA.
  • I listed at least one issue that this PR fixes in the description above.
  • I updated/added relevant documentation (doc comments with ///).
  • I added new tests to check the change I am making, or this PR is test-exempt.
  • All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

@flutter-dashboard flutter-dashboard Bot added the tool Affects the "flutter" command-line tool. See also t: labels. label Dec 7, 2022
@flutter-dashboard

Copy link
Copy Markdown

It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption to this rule, contact Hixie on the #hackers channel in Chat (don't just cc him here, he won't see it! He's on Discord!).

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

@christopherfujino christopherfujino left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM. Your CI tests are failing from an infra issue, and I think you'll need to add tests.

@christopherfujino
christopherfujino self-requested a review December 8, 2022 17:37
@nehalvpatel

nehalvpatel commented Dec 8, 2022 •

Copy link
Copy Markdown
Contributor Author

Sounds good, I'll add the test(s).

@nehalvpatel nehalvpatel changed the title Fix flutter update-packages regression by fixing parameters in "pub get" runner Draft: Fix flutter update-packages regression by fixing parameters in "pub get" runner Dec 13, 2022
@nehalvpatel
nehalvpatel marked this pull request as draft December 13, 2022 23:20
@flutter-dashboard

Copy link
Copy Markdown

This pull request has been changed to a draft. The currently pending flutter-gold status will not be able to resolve until a new commit is pushed or the change is marked ready for review again.

For more guidance, visit Writing a golden file test for package:flutter.

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

@christopherfujino

Copy link
Copy Markdown
Contributor

cc @andrewkolos

@nehalvpatel
nehalvpatel marked this pull request as ready for review December 14, 2022 01:26
@nehalvpatel nehalvpatel changed the title Draft: Fix flutter update-packages regression by fixing parameters in "pub get" runner Fix flutter update-packages regression by fixing parameters in "pub get" runner Dec 14, 2022
@nehalvpatel

Copy link
Copy Markdown
Contributor Author

Looks like there's something in this PR that causes all of the following tests to time out. They're all Windows tests. It's specifically on the "download dependencies" step.

Example of success from a different PR: https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8794762758857728449/+/u/download_dependencies/stdout

Example of failure: https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8794777666714712193/+/u/download_dependencies/stdout

  • The last line is from the infra tooling automatically cancelling the task after it stalled for 3 hours.

I believe it's this last line in flutter.bat where it gets stuck:

"%dart%" --disable-dart-dev --packages="%flutter_tools_dir%\.dart_tool\package_config.json" %FLUTTER_TOOL_ARGS% "%snapshot_path%" %* & "%exit_with_errorlevel%"


The tests:

@christopherfujino

Copy link
Copy Markdown
Contributor

Looks like there's something in this PR that causes all of the following tests to time out. They're all Windows tests. It's specifically on the "download dependencies" step.

Example of success from a different PR: https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8794762758857728449/+/u/download_dependencies/stdout

Example of failure: https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8794777666714712193/+/u/download_dependencies/stdout

  • The last line is from the infra tooling automatically cancelling the task after it stalled for 3 hours.

I believe it's this last line in flutter.bat where it gets stuck:

"%dart%" --disable-dart-dev --packages="%flutter_tools_dir%\.dart_tool\package_config.json" %FLUTTER_TOOL_ARGS% "%snapshot_path%" %* & "%exit_with_errorlevel%"

The tests:

Taking a look now

@christopherfujino

Copy link
Copy Markdown
Contributor
Checking Dart SDK version... 
Downloading Dart SDK from Flutter engine ... 
Expanding downloaded archive...
Building flutter tool... 
Running pub upgrade... 
Downloading Material fonts...                                      510ms
Downloading Gradle Wrapper...                                       12ms
Downloading package sky_engine...                                  116ms
Downloading flutter_patched_sdk tools...                           305ms
Downloading flutter_patched_sdk_product tools...                   300ms
Downloading windows-x64 tools...                                 2,247ms
Downloading windows-x64/font-subset tools...                       135ms
^CTerminate batch job (Y/N)? 

So what's happening here is that LUCI is trying to run flutter update-packages. In order, what happens is:

  1. the flutter.bat batch script downloads the Dart SDK
  2. the flutter.bat script compiles the flutter tool
  3. the flutter.bat script runs dart pub upgrade in the flutter_tools directory
  4. the flutter.bat script invokes the flutter tool with update-packages as the sub-command
  5. the tool starts caching pre-built binaries required by the update-packages sub-command (does this finish and the next command start?)
  6. the tool hangs until LUCI times it out after 3 hours

Given the logs in the succeeding step you linked, I'm guessing step 5 finished. I'm not sure exactly what happened, I think you need to investigate more, but maybe your change made the tool stop logging "Running pub upgrade...", then invoked pub. Then, for whatever reason, it's retrying forever until it gets timed out?

One thing you could do is hack the tool to always run this sub-command in verbose mode just for your PR, push it to github, and then look at the verbose LUCI logs.

@nehalvpatel

Copy link
Copy Markdown
Contributor Author

Thanks for taking a look! I'll give that a shot.

@nehalvpatel

nehalvpatel commented Dec 17, 2022 •

Copy link
Copy Markdown
Contributor Author

@christopherfujino

Looks like the issue was fixed.

Apparently it was because I was using ProcessUtils.start for the non-output mode, which I now see does warn you in the documentation to make sure to read all the data from the stdout / stderr streams, but as of now only trips up on Windows if you don't do that. I left a comment about it in pub.dart. 😵‍💫

This issue and comment clued me to it: dart-lang/sdk#50689 (comment)

@auto-submit
auto-submit Bot merged commit de2a424 into flutter:master Jan 6, 2023
engine-flutter-autoroll added a commit to engine-flutter-autoroll/plugins that referenced this pull request Jan 6, 2023
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jan 6, 2023
auto-submit Bot pushed a commit to flutter/packages that referenced this pull request Jan 6, 2023
* 7a5ae7c Roll Plugins from f9dda6a27b79 to 320461910156 (2 revisions) (flutter/flutter#118040)

* 43b9120 072a9ca37 Add `TextProvider` and `TextEdit` patterns to `AXPlatformNodeWin` (flutter-team-archive/engine#38646) (flutter/flutter#118039)

* 5a87a82 bb4015269 Roll Skia from 158d51b34caa to ecd3a2f865ba (1 revision) (flutter-team-archive/engine#38659) (flutter/flutter#118042)

* 0a2e0a4 Avoid using `TextAffinity` in `TextBoundary` (flutter/flutter#117446)

* 36a1caf 74861f369 Reduce the size of Overlay FlutterImageView in HC mode (flutter-team-archive/engine#38393) (flutter/flutter#118048)

* 9b864b8 5bd90d6e7 Consider more roles as text (flutter-team-archive/engine#38645) (flutter/flutter#118049)

* 70d5834 [EMPTY] Commit to refresh the tree that is currently red (flutter/flutter#118062)

* 4f3ed80 Remove doc reference to the deprecated ui.FlutterWindow API (flutter/flutter#118064)

* de2a424 Fix `flutter update-packages` regression by fixing parameters in "pub get" runner (flutter/flutter#116687)

* 57dc071 Adding 'is' to list of kotlin reserved keywords (flutter/flutter#116299)

* 5070620 Added expandIconColor property on ExpansionPanelList Widget (flutter/flutter#115950)
auto-submit Bot pushed a commit to flutter-team-archive/plugins that referenced this pull request Jan 6, 2023
* 5a87a82 bb4015269 Roll Skia from 158d51b34caa to ecd3a2f865ba (1 revision) (flutter-team-archive/engine#38659) (flutter/flutter#118042)

* 0a2e0a4 Avoid using `TextAffinity` in `TextBoundary` (flutter/flutter#117446)

* 36a1caf 74861f369 Reduce the size of Overlay FlutterImageView in HC mode (flutter-team-archive/engine#38393) (flutter/flutter#118048)

* 9b864b8 5bd90d6e7 Consider more roles as text (flutter-team-archive/engine#38645) (flutter/flutter#118049)

* 70d5834 [EMPTY] Commit to refresh the tree that is currently red (flutter/flutter#118062)

* 4f3ed80 Remove doc reference to the deprecated ui.FlutterWindow API (flutter/flutter#118064)

* de2a424 Fix `flutter update-packages` regression by fixing parameters in "pub get" runner (flutter/flutter#116687)

* 57dc071 Adding 'is' to list of kotlin reserved keywords (flutter/flutter#116299)

* 5070620 Added expandIconColor property on ExpansionPanelList Widget (flutter/flutter#115950)
loic-sharma pushed a commit to fluttergithubbot/flutter that referenced this pull request Jan 6, 2023
… get" runner (flutter#116687)

* Make pub get runner respect printProgress and retry parameters

* Fix typo

* Add regression test

* Improve test

* Fix implementation and test

* Test to fix flutter_drone tests

* Revert test

* Attempt #2 to fix flutter_drone tests

* Revert attempt

* Hack: Force printProgress to debug Windows tests

* Use ProcessUtils.run to avoid dangling stdout and stderr

* Update documentation

* Clean up retry argument
esouthren pushed a commit to esouthren/flutter that referenced this pull request Jan 15, 2023
… get" runner (flutter#116687)

* Make pub get runner respect printProgress and retry parameters

* Fix typo

* Add regression test

* Improve test

* Fix implementation and test

* Test to fix flutter_drone tests

* Revert test

* Attempt #2 to fix flutter_drone tests

* Revert attempt

* Hack: Force printProgress to debug Windows tests

* Use ProcessUtils.run to avoid dangling stdout and stderr

* Update documentation

* Clean up retry argument
esouthren pushed a commit to esouthren/flutter that referenced this pull request Jan 16, 2023
… get" runner (flutter#116687)

* Make pub get runner respect printProgress and retry parameters

* Fix typo

* Add regression test

* Improve test

* Fix implementation and test

* Test to fix flutter_drone tests

* Revert test

* Attempt #2 to fix flutter_drone tests

* Revert attempt

* Hack: Force printProgress to debug Windows tests

* Use ProcessUtils.run to avoid dangling stdout and stderr

* Update documentation

* Clean up retry argument