-
Notifications
You must be signed in to change notification settings - Fork 29.7k
Closed
flutter/engine
#50826Closed
Copy link
Labels
P1High-priority issues at the top of the work listHigh-priority issues at the top of the work listc: proposalA detailed proposal for a change to FlutterA detailed proposal for a change to Flutterc: tech-debtTechnical debt, code quality, testing, etc.Technical debt, code quality, testing, etc.engineflutter/engine related. See also e: labels.flutter/engine related. See also e: labels.fyi-androidFor the attention of Android platform teamFor the attention of Android platform teamfyi-iosFor the attention of iOS platform teamFor the attention of iOS platform teamfyi-webFor the attention of Web platform teamFor the attention of Web platform teamteam-engineOwned by Engine teamOwned by Engine teamtriaged-engineTriaged by Engine teamTriaged by Engine team
Description
🚫 Blocking #143352.
In April 2023, we made the SkiaGold check non-blocking with the assumption the engine sheriff would review them.
In practice, that has not (IMO) worked out particularly well:
- The sheriff is poorly suited to make decisions other than "this needs further triage" or not
- Lack of a blocking tree means its easy to introduce unstable or even broken golden tests
- Finding the right person to review and act on untriaged goldens a time consuming and error-prone task.
We need to decide:
- Do we want to block the tree on untriaged goldens
- If yes, how do we deal with tree breakages due to untriaged goldens
The rest of this issue assumes "yes", but please chime in if you think the answer should be "no".
Questions to answer:
- What if a team wants to "try" goldens on CI, without committing to them?
- Do we have "bringup"-style goldens?
- Do we consider either a dimension that means "can fail" or even a separate tree (
flutter-engine-bringup.skiagold)?
- What if a golden is flaky, i.e. a slightly different image is emitted and Skia gold can't stabilize?
- What if the tree somehow makes it to a red state out of band (i.e. there is no obvious PR to revert)?
- Should the sheriff be disabling failing/flaky golden tests in order to bring the tree green?
@zanderso @dnfield @gaaclarke @jmagman @reidbaker @johnmccutchan @jonahwilliams @Piinks @yjbanov @mdebbar
Metadata
Metadata
Assignees
Labels
P1High-priority issues at the top of the work listHigh-priority issues at the top of the work listc: proposalA detailed proposal for a change to FlutterA detailed proposal for a change to Flutterc: tech-debtTechnical debt, code quality, testing, etc.Technical debt, code quality, testing, etc.engineflutter/engine related. See also e: labels.flutter/engine related. See also e: labels.fyi-androidFor the attention of Android platform teamFor the attention of Android platform teamfyi-iosFor the attention of iOS platform teamFor the attention of iOS platform teamfyi-webFor the attention of Web platform teamFor the attention of Web platform teamteam-engineOwned by Engine teamOwned by Engine teamtriaged-engineTriaged by Engine teamTriaged by Engine team