-
Notifications
You must be signed in to change notification settings - Fork 29.7k
[github actions] Automate Flutter Chery Picks #139524
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
CaseyHillers
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Can we create a flutter.dev/go/ link so anyone can access the design doc?
- Can we link to the tracking bug?
| steps: | ||
| - name: Get Release Channel | ||
| run: | | ||
| echo "CHANNEL=$(echo ${{ github.event.label.name }} | cut -d ':' -f 2 | xargs)" >> $GITHUB_ENV |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the double echo unnecessary?
export CHANNEL=$(${{ github.event.label.name }} | cut -d ':' -f 2)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tested and my understanding is that both the inner and outer echos are needed.
My understanding is that the inner echo is needed to pipe it to the next expression.
without the inner echo, $word | cut -d ':' -f 2 would throw command not found error on $word
with the echo, echo $word | cut -d ':' -f 2 would pipe the output of echo $word into the next expression
The outer echo is needed due to the unique way of storing environment variables in github actions.
without the outer echo and using export $var, the variable is not stored in github actions. failed workflow
with the outer echo, Echo it into GITHUB_ENV would prevserve it. (https://stackoverflow.com/questions/57968497/how-do-i-set-an-env-var-with-a-bash-expression-in-github-actions)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To help with future reviews, do you have a recommended resource on the shell syntax GitHub is using in these run fields?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Umm yeah I haven't found a source that includes all the supported syntax. Currently my learnings come from failed workflows and stackoverflow.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.github/workflows/easy-cp.yml
Outdated
| git checkout -b cp-$COMMIT_SHA --track origin/$RELEASE_BRANCH | ||
| git cherry-pick $COMMIT_SHA | ||
| git status | ||
| # TODO(xilaizhang): move cp template to flutter/flutter. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this be implemented now?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, created #139590.
The template has to be in markdown format cause the original web form issue template (https://github.com/flutter/flutter/blob/master/.github/ISSUE_TEMPLATE/7_cherry_pick.yml) isn't supported in PR description.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For future reference, I think we could've implemented that PR in this one to keep all the context together.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah I implemented separately cause I thought if the template gets merged into flutter/flutter first, then we can directly use it in github actions. otherwise even if they are in the same PR, the github action still has to download the template from somewhere else because the template doesn't exist on framework yet.
.github/workflows/easy-cp.yml
Outdated
| with: | ||
| base: ${{ env.RELEASE_BRANCH }} | ||
| path: flutter | ||
| token: ${{ secrets.PAT }} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of using a PAT, is the plan to follow https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs to grant write access to this job's GITHUB_TOKEN?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah we have the line permissions: write-all.
Good point! did some tests and I think it works on the same repo where the action is installed. e.g. the github.token of flutter/flutter cannot comment on flutter/engine. but should work on flutter/flutter after we merge this PR.
.github/workflows/easy-cp.yml
Outdated
| base: ${{ env.RELEASE_BRANCH }} | ||
| path: flutter | ||
| token: ${{ secrets.PAT }} | ||
| commit-message: automated cherry pick |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we re-use the original commit message?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My understanding is that there isn't an original commit message, because the message associated with a merged commit sha (cp commit sha), is the PR description of the merged PR.
e.g. the merged commit sha is af61c6c of #139348. running the query gh search commits --repo flutter/flutter --hash e5762ba --json commit gives a commit message that is several hundred chars (the pr description)
e.g. in the cherry pick pr #139594. The commit message associated with the cherry picked commit is the original PR title.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That SGTM as that's expected. However, is commit message needed here? It seems to be creating an empty commit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, removed the commit-message and committer fields.
I think by default their values are set by the market place action to be [create-pull-request] automated change and GitHub <[email protected]> if we omit these fields
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need the create pull request action? Can we use the gh cli directly?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah we can definitely use gh cli directly.
My first few drafts are all implemented with pure gh/git commands, but I think in previous design/code reviews it was pointed out that I should use market place actions whenever possible. I am good with reimplementing with gh and git commands again.
However, if the concern here is that an unnecessary commit message will be generated, then I can assure that no duplicate/extra commit message will be produced. See #139843 as an example where I cp-ed a chillers pr using market place action.
| git status | ||
| # TODO(xilaizhang): move cp template to flutter/flutter. | ||
| # The Pull Request cherry pick template should be under flutter/.github/PULL_REQUEST_CP_TEMPLATE.md. | ||
| - name: Get CP Template |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this needed if below we use "body-path" to use the template?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah i added a TODO here but we can only remove this part after #139590 lands (or when Kevin is back).
Without an existing template in flutter/flutter, we will have to download the template from somewhere and include it in the opened pull request.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With #139590 ready, can we remove the TODO?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tested around and realized I cannot remove this step because the template isn't on release branches yet.
Currently the template is only on master head and not on release branches. When we are creating the pull request from the release branch, this file path becomes unavailable to us. For now, I still have to download it from the flutter/flutter repo.
Sure, created a tracking bug at #139604. |
| steps: | ||
| - name: Get Release Channel | ||
| run: | | ||
| echo "CHANNEL=$(echo ${{ github.event.label.name }} | cut -d ':' -f 2 | xargs)" >> $GITHUB_ENV |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To help with future reviews, do you have a recommended resource on the shell syntax GitHub is using in these run fields?
.github/workflows/easy-cp.yml
Outdated
| token: ${{ github.token }} | ||
| path: flutter | ||
| ref: master | ||
| # Use depth 0 to checkout history of master branch, so the cp commit is a known object |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems counter-intuitive to do a depth 0 to ensure we have the CP object. Is depth needed in this case? I thought we were doing it for space savings on the runner.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah so fetch-depth means the number of commits to fetch, and fetch-depth: x means fetching the last x commits on branch ref.
For the cherry pick commit, its unclear where the commit is on the master branch. e.g. the cp commit could be located at 201 commits before HEAD, and therefore fetch-depth: 200 would not retrieve the cp commit.
I had to do a fetch-depth:0 here (0 fetches all commits on ref), so that no matter where the cp commit is in the commit history, it is always retrieved and a known object. Even if it is the first commit of the repository.
By default, fetch-depth is set to 1 which only retrieves the tip commit on branch ref.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In that case, updating the docs to say "to checkout the history of master branch" would make this clear. In it's current state, it's still implied it's getting 0 history.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated the comment to Checkout all history commits on master branch, so that the cp commit is a known object
I might have misunderstood and I am good with whichever comment that would make this part clearer. If Casey proposes a change here I d be happy to accept it.
.github/workflows/easy-cp.yml
Outdated
| base: ${{ env.RELEASE_BRANCH }} | ||
| path: flutter | ||
| token: ${{ secrets.PAT }} | ||
| commit-message: automated cherry pick |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That SGTM as that's expected. However, is commit message needed here? It seems to be creating an empty commit.
.github/workflows/easy-cp.yml
Outdated
| git checkout -b cp-$COMMIT_SHA --track origin/$RELEASE_BRANCH | ||
| git cherry-pick $COMMIT_SHA | ||
| git status | ||
| # TODO(xilaizhang): move cp template to flutter/flutter. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For future reference, I think we could've implemented that PR in this one to keep all the context together.
Please attach it to the pull request description above. For cases like this, it's helpful to keep the standard pull request template to make sure the PR adheres to the repos guidelines. (see https://github.com/flutter/flutter/blob/master/.github/PULL_REQUEST_TEMPLATE.md)
|
…on (#139590) context: #139524 (comment) Adds a pull request template for automated flutter cherry picks. This template is adapted from [web form cherry pick issue template](https://cs.opensource.google/flutter/flutter/+/master:.github/ISSUE_TEMPLATE/7_cherry_pick.yml) since PR template doesn't support web form. This would provide the template to allow #139524 to land
Sure, attached Oh TIL, created an issue at #139846 |
|
Do we know why Google testing is stuck in the queued state for 2+ days? |
Roll Flutter from 9719097 to 11a9cb7 (32 revisions) flutter/flutter@9719097...11a9cb7 2023-12-13 [email protected] Make tests more resilient to Skia gold failures and refactor flutter_goldens for extensive technical debt removal (flutter/flutter#139549) 2023-12-12 49699333+dependabot[bot]@users.noreply.github.com Bump github/codeql-action from 2.22.9 to 2.22.10 (flutter/flutter#140003) 2023-12-12 [email protected] Allow plugins to use compileSdkPreview (flutter/flutter#131901) 2023-12-12 [email protected] Roll pub packages (flutter/flutter#139995) 2023-12-12 [email protected] Select simulator runtime for tests based on Xcode's preferred runtime build (flutter/flutter#139919) 2023-12-12 [email protected] Roll Flutter Engine from 0c527aa1a215 to 9039ac78cf03 (1 revision) (flutter/flutter#139992) 2023-12-12 [email protected] [Docs] Added missing `CupertinoApp.showSemanticsDebugger` (flutter/flutter#139913) 2023-12-12 [email protected] Roll Flutter Engine from 444281eb5c7c to 0c527aa1a215 (2 revisions) (flutter/flutter#139985) 2023-12-12 [email protected] Update Gallery lockfiles for the new version of the video_player plugin (flutter/flutter#139832) 2023-12-12 [email protected] Roll Packages from cb6dbcd to 80aa46a (5 revisions) (flutter/flutter#139982) 2023-12-12 [email protected] Roll Flutter Engine from 3b77b1b7b42f to 444281eb5c7c (1 revision) (flutter/flutter#139979) 2023-12-12 [email protected] Roll Flutter Engine from f8e87ed193f5 to 3b77b1b7b42f (1 revision) (flutter/flutter#139977) 2023-12-12 [email protected] Roll pub packages (flutter/flutter#139969) 2023-12-12 [email protected] Roll Flutter Engine from 4102c7daf1d3 to f8e87ed193f5 (3 revisions) (flutter/flutter#139963) 2023-12-12 [email protected] Roll Flutter Engine from 95dfb1d4ac75 to 4102c7daf1d3 (1 revision) (flutter/flutter#139961) 2023-12-12 [email protected] Roll Flutter Engine from 40bfd2dc1519 to 95dfb1d4ac75 (1 revision) (flutter/flutter#139959) 2023-12-12 [email protected] Roll Flutter Engine from 061ae7023f10 to 40bfd2dc1519 (2 revisions) (flutter/flutter#139958) 2023-12-12 [email protected] Roll Flutter Engine from 75cfb050cd9a to 061ae7023f10 (1 revision) (flutter/flutter#139955) 2023-12-12 [email protected] Roll Flutter Engine from 362d0cb3ab27 to 75cfb050cd9a (1 revision) (flutter/flutter#139954) 2023-12-12 [email protected] Fix dayPeriodColor handling of non-MaterialStateColors (flutter/flutter#139845) 2023-12-12 [email protected] Roll Flutter Engine from ea1a3069e057 to 362d0cb3ab27 (1 revision) (flutter/flutter#139951) 2023-12-12 [email protected] [github actions] Automate Flutter Chery Picks (flutter/flutter#139524) 2023-12-12 [email protected] Roll Flutter Engine from d001419b436e to ea1a3069e057 (5 revisions) (flutter/flutter#139948) 2023-12-12 [email protected] [flutter_tools] catch SocketException writing to ios-deploy stdin (flutter/flutter#139784) 2023-12-11 [email protected] [ci.yaml] Add missing ci.yaml to runIf of android hot reload tests (flutter/flutter#139932) 2023-12-11 [email protected] make the tar c command in prepare_package.dart verbose (flutter/flutter#139687) 2023-12-11 [email protected] Roll Flutter Engine from 5c1f13e1e535 to d001419b436e (4 revisions) (flutter/flutter#139941) 2023-12-11 [email protected] fix typo of 'not' instead of 'now' for `useInheritedMediaQuery` (flutter/flutter#139940) 2023-12-11 [email protected] Roll Flutter Engine from 4c309195b79d to 5c1f13e1e535 (2 revisions) (flutter/flutter#139939) 2023-12-11 [email protected] Implement `switch` expressions in `examples/` and `animation/` (flutter/flutter#139882) 2023-12-11 [email protected] Renamed `appbar` to `app_bar` directory in API Examples Tests (flutter/flutter#139922) 2023-12-11 [email protected] Deprecate `RawKeyEvent`, `RawKeyboard`, et al. (flutter/flutter#136677) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC [email protected],[email protected],[email protected] on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md ...
design doc: go/easy-cp
umbrella design doc: go/flutter-actions
umbrella bug: #139604
Sample Results:
[beta-cherrypick] cherrypicks commit 7d9010c3578338bc850d66c315bd6086f3d3dc9a from PR #139523
[scheduler] Add recipe property to support releases cocoon#3305
In tests, 7d9010 was used to simulate a clean cherry pick, and cf71a5 was used to simulate a merge conflict during cherry pick.
Implementation Details:
PR template
Since PR template doesn't support web form, the cherry pick PR template we used is https://github.com/XilaiZhang/miscellaneous-side-project/blob/master/.github/workflows/template/cherrypick.md, which is adapted from the web form cherry pick issue template.
This PR template should be reviewed and added to be under the .github path in framework repository.
Decisions Taken
place of source code
I put the source code under the framework repository. Acknowledges the risk of duplication of code, in favor of not needing to package and publish our own actions to market place.
more details in open discussions section of design doc
Resolution strategy
No resolution strategy is applied at all during the cherry pick process. more details
Future Work
A PAT token is needed for authorization to create branch and add comment. We would need to create a new bot and update its credentials in the secrets section of flutter/flutter repository.
Pre-launch Checklist
///).If you need help, consider asking for advice on the #hackers-new channel on Discord.