Skip to content

Add golden test review process to release cycle #144835

@gaaclarke

Description

@gaaclarke

Currently, when we create release builds of the engine, the golden image test results are not checked (automated or manually). Some of them are not even executed, only certain invocations of "run_tests.py" are used.

We'd like to have confidence that our cherry-picks are not causing some failure, so ideally the golden tests would be executed and checked.

The golden image results can not be naively uploaded to skia gold because skia gold has a consistent timeline that checks against one branch. Comparing a golden from a release branch to the main branch would confuse skia gold and lead to situations where there is no correct course forward.

A potential solution would be to create new tests related to the release branch. For example aiks_foo_test.png could become aiks_foo_test.png.v12_34_5 for release 12.32.5. The tests would first be uploaded with zero cherry-picks and blanket approved. Those images will then serve as the basis from which cherry-pick golden results will be compared.

cc @zanderso @matanlurey

Metadata

Metadata

Assignees

Labels

P2Important issues not at the top of the work listc: proposalA detailed proposal for a change to Flutterteam-engineOwned by Engine teamtriaged-engineTriaged by Engine team

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions