description
It's becoming difficult to manage what exists in devicelab tests already. This requires people to have to have memorized what exists or to guess what exists based on filenames. For example: "do we have a windows benchmark measuring startup time with impeller?"
This is also problematic for ai agents since they have to parse Dart code and made deductions from filenames as well.
This is a proposal to get all the devicelab tests documented in markdown which makes it easier to find what exists already in device lab
details
- Move the TESTOWNERS validation check into the flutter/flutter repo (right now it's in recipes, we already did this for the ci.yaml validator)
- Expand TESTOWNERS to include a description of the test and links to generated benchmarks
- Expand the TESTOWNERS validation check to make sure there is a description for everything
- Task an agent to backfill descriptions
alternatives considered
1) create a new markdown file with test descriptions
Instead of expanding this data to live in TESTOWNERS we could have created a new markdown file that lists tests descriptions. I decided against this since we already have team muscle memory to update this file and automated tools to do it. It seems preferable to expand that instead of creating a whole new thing.
description
It's becoming difficult to manage what exists in devicelab tests already. This requires people to have to have memorized what exists or to guess what exists based on filenames. For example: "do we have a windows benchmark measuring startup time with impeller?"
This is also problematic for ai agents since they have to parse Dart code and made deductions from filenames as well.
This is a proposal to get all the devicelab tests documented in markdown which makes it easier to find what exists already in device lab
details
alternatives considered
1) create a new markdown file with test descriptions
Instead of expanding this data to live in TESTOWNERS we could have created a new markdown file that lists tests descriptions. I decided against this since we already have team muscle memory to update this file and automated tools to do it. It seems preferable to expand that instead of creating a whole new thing.