You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[cocoon] Not reporting "success" after retrying a build. #116226
I retried manually here through the Cocoon "RERUN" button. That set the Github status "in progress" again, but once the build successfully passed, the Github issue never went green again. Instead, it's "updating" every few minutes, but it never goes to green.
Flutter Ecosystem tooling relies on the tree being all green for CI automation to keep going. For Flutter Rolls this is not a big issue, because they never trigger a release of any package, but if a feature PR lands and ends up in this state, our release procedure must be performed manually (we have no automation override), and it's sub-optimal.
Can Cocoon report successful retried builds to Github as a success? Is this not a Cocoon problem and are we doing something wrong?
For my understanding, did Cocoon report successful retried builds to Github in the past and is failing to now, or is this a request for an additional feature?
/cc @CaseyHillers potentially, who would know more than me
@ditman can you screenshot what you're clicking? I see "Retry Build" in that link, which does not work. You need to use flutter-dashboard.appspot.com to retry postsubmit builds.
We could add support for the postsubmit GitHub checks rerun functionality as well.
That updated the github status to "in progress" (as expected), and kicked off the LUCI build, but when the build finished successfully, the success was not reported to the github run.
(If you take a look at the run now, github thinks it started 5 minutes ago (??) This seems to attempt to refresh every 5 or 6 minutes?)
@stuartmorgan Do you know if this feature would be a part of that work?
I wasn't aware that there was any remaining issue with the post-submit checkin; as far as I knew it was done. The expectation we set was was that post-submit reporting from LUCI was a precondition for migration of any CI to LUCI, so I'm presuming Keyong didn't know that this didn't work either.
(Asking since it looks like you are taking over that work from Keyong)
The plan was for Keyong to come back to finish the migration; there were a few individual tests I was considering migrating in the meantime, but that's it.
This issue puts us in a bad place: we can't revert the migration because the Cirrus macOS Intel support that we migrated these tasks away from will be eliminated on January 1st, and we can't migrate the tasks to Cirrus ARM because of bugs in the Cirrus ARM hosts that haven't been resolved, but the current situation will cause us a lot of manual pain for the team due to the disruption to autoreleases described above. Is there anyone else with Cocoon expertise who could look into this bug in the meantime?
We may need to refactor the entire data models soon, adding new columns will complicate that migration. Do we know what the issue is? is there a way to fix it without adding new columns to data models?
I discussed with Godofredo on chat and he suggested a different approach. I'll be working on that. Anywho, I don't want to turn this thread into an implementation discussion so I'll make a new issue. 😄
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.
Activity
drewroengoogle commentedon Nov 30, 2022
For my understanding, did Cocoon report successful retried builds to Github in the past and is failing to now, or is this a request for an additional feature?
/cc @CaseyHillers potentially, who would know more than me
CaseyHillers commentedon Nov 30, 2022
@ditman can you screenshot what you're clicking? I see "Retry Build" in that link, which does not work. You need to use flutter-dashboard.appspot.com to retry postsubmit builds.
We could add support for the postsubmit GitHub checks rerun functionality as well.
ditman commentedon Nov 30, 2022
@CaseyHillers I clicked this button:
That updated the github status to "in progress" (as expected), and kicked off the LUCI build, but when the build finished successfully, the success was not reported to the github run.
(If you take a look at the run now, github thinks it started 5 minutes ago (??) This seems to attempt to refresh every 5 or 6 minutes?)
drewroengoogle commentedon Dec 1, 2022
I talked with Casey a bit and it sounds like this may be a feature that Keyong was working on that is a part of moving the plugin team to LUCI.
@stuartmorgan Do you know if this feature would be a part of that work? (Asking since it looks like you are taking over that work from Keyong)
ditman commentedon Dec 1, 2022
@drewroengoogle how is this different from Cocoon reporting build successes when I don't have to restart the build by hand? That bit works fine :/
CaseyHillers commentedon Dec 1, 2022
Up until recently, Cocoon never supported GitHub checks for postsubmit. This sounds like a bug as the feature is still in development.
ditman commentedon Dec 2, 2022
Roger that! Thanks for the details @CaseyHillers!
drewroengoogle commentedon Dec 2, 2022
@CaseyHillers Does it make sense to keep this issue open but move it out of the queue based the info you've provided?
stuartmorgan-g commentedon Dec 3, 2022
I wasn't aware that there was any remaining issue with the post-submit checkin; as far as I knew it was done. The expectation we set was was that post-submit reporting from LUCI was a precondition for migration of any CI to LUCI, so I'm presuming Keyong didn't know that this didn't work either.
The plan was for Keyong to come back to finish the migration; there were a few individual tests I was considering migrating in the meantime, but that's it.
This issue puts us in a bad place: we can't revert the migration because the Cirrus macOS Intel support that we migrated these tasks away from will be eliminated on January 1st, and we can't migrate the tasks to Cirrus ARM because of bugs in the Cirrus ARM hosts that haven't been resolved, but the current situation will cause us a lot of manual pain for the team due to the disruption to autoreleases described above. Is there anyone else with Cocoon expertise who could look into this bug in the meantime?
CaseyHillers commentedon Dec 5, 2022
@ricardoamador or @nehalvpatel can you take a look?
nehalvpatel commentedon Dec 5, 2022
Taking a look.
nehalvpatel commentedon Dec 7, 2022
I think this will require the datastore
Taskentity to be updated with a new column forcheck_run_id. It looks like the LUCI build doesn't store it (maybe we can put it in there?). https://cr-buildbucket.appspot.com/rpcexplorer/services/buildbucket.v2.Builds/GetBuild?request={%20%20%20%20%22id%22:%20%228796130005986382449%22}Does that sound right, @CaseyHillers?
CaseyHillers commentedon Dec 7, 2022
SGTM!
godofredoc commentedon Dec 7, 2022
We may need to refactor the entire data models soon, adding new columns will complicate that migration. Do we know what the issue is? is there a way to fix it without adding new columns to data models?
nehalvpatel commentedon Dec 8, 2022
I discussed with Godofredo on chat and he suggested a different approach. I'll be working on that. Anywho, I don't want to turn this thread into an implementation discussion so I'll make a new issue. 😄
nehalvpatel commentedon Jan 23, 2023
Awaiting final review on this PR which should fix this: flutter/cocoon#2387
nehalvpatel commentedon Jan 26, 2023
This should be fixed now. flutter/cocoon#2387 (comment)
Please reopen if the issue continues!
ricardoamador commentedon Jan 26, 2023
Nice work @nehalvpatel !
ditman commentedon Feb 1, 2023
Have you ever been excited about a new CI feature? Because I'm excited about a new CI feature! Thanks people!! 🤟
nehalvpatel commentedon Feb 1, 2023
Thank you for the kind words :)
github-actions commentedon Mar 3, 2023
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of
flutter doctor -vand a minimal reproduction of the issue.