-
Notifications
You must be signed in to change notification settings - Fork 29.7k
Description
Currently, when we create release builds of the engine, the golden image test results are not checked (automated or manually). Some of them are not even executed, only certain invocations of "run_tests.py" are used.
We'd like to have confidence that our cherry-picks are not causing some failure, so ideally the golden tests would be executed and checked.
The golden image results can not be naively uploaded to skia gold because skia gold has a consistent timeline that checks against one branch. Comparing a golden from a release branch to the main branch would confuse skia gold and lead to situations where there is no correct course forward.
A potential solution would be to create new tests related to the release branch. For example aiks_foo_test.png could become aiks_foo_test.png.v12_34_5 for release 12.32.5. The tests would first be uploaded with zero cherry-picks and blanket approved. Those images will then serve as the basis from which cherry-pick golden results will be compared.