Repository navigation
Commit 7a62b1f
authored
Run 10 impeller iOS benchmarks as one devicelab task (flutter#195661)
draw_points, draw_vertices, draw_atlas, draw_arcs_all_fill_styles,
draw_arcs_all_stroke_styles, rrect_blur, and
*_perf_ios__timeline_summary tasks all build the same macrobenchmarks
app with the same target and each draw for about 10 seconds. Each spends
1.4–2.6 minutes benchmarking inside a 5–10 minute bot run. This runs
them in one task,
`impeller_primitives_perf_suite_ios__timeline_summary`.
Each benchmark still gets its own `flutter drive`, so the app is
reinstalled and relaunched every time and nothing in-process carries
over. Runs after the first reuse the already-built .app so the app is
only built once.
SkiaPerf traces are unchanged. TaskResult.merge prefixes each score key
with the original task name and `metrics_center.parse` uploads it under
that name. Tasks that don't use merge are unaffected.
Per commit **this runs a 16 minutes benchmark on one Mac and iPhone
instead of 76 minutes across 10 benchmarks**, saving ~100 device-hours a
week. The suite becomes the longest Mac_ios task (~16 min), but these
are postsubmit. Failure in any benchmark fails the task and drops that
commit's metrics for all 10 benchmarks. The task runs every benchmark
and reports all failures, not just the first.
Delete the old builders (which also means release branch cp). Ideally we
could run them next to each other for awhile, but unfortunately the way
SkiaPerf works the metrics are the same, so they would stomp on each
other and we wouldn't know which builder the data is coming from (they
even upload in `bringup`). Let's see if any of the metrics move on this
commit, if they do we can go back.
[Test run
build](https://ci.chromium.org/ui/p/flutter/builders/try/Mac_ios%20draw_arcs_all_fill_styles_perf_ios__timeline_summary/1/overview)
- metrics at end of
[stdout](https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8666540421791185473/+/u/run_impeller_primitives_perf_suite_ios__timeline_summary/stdout)
- 14m total, 9.5m on the actual benchmark run step.
Part of the work to reduce the number of Mac bots in CI.
flutter#195654
## Pre-launch Checklist
- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [AI contribution guidelines] and understand my
responsibilities, or I am not using AI tools.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant in-code documentation (doc comments with
`///`).
- [x] If this PR introduces a new feature or capability, I created and
linked a website documentation issue or PR in [flutter/website] (or
verified none is needed).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [x] All existing and new tests are passing.
If you need help, consider asking for advice on the #hackers-new channel
on [Discord].
If this change needs to override an active code freeze, provide a
comment explaining why. The code freeze workflow can be overridden by
code reviewers. See pinned issues for any active code freezes with
guidance.
**Note**: The Flutter team is currently trialing the use of [Gemini Code
Assist for
GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code).
Comments from the `gemini-code-assist` bot should not be taken as
authoritative feedback from the Flutter team. If you find its comments
useful you can update your code accordingly, but if you are unsure or
disagree with the feedback, please feel free to wait for a Flutter team
member's review for guidance on which automated comments should be
addressed.
<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview
[AI contribution guidelines]:
https://github.com/flutter/flutter/blob/main/CONTRIBUTING.ai.md
[Tree Hygiene]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
[test-exempt]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[flutter/website]: https://github.com/flutter/website
[breaking change policy]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes
[Discord]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md
[Data Driven Fixes]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md1 parent 852534a commit 7a62b1f
19 files changed
Lines changed: 204 additions & 235 deletions
File tree
- dev/devicelab
- bin/tasks
- lib/framework
- test
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3759 | 3759 | | |
3760 | 3760 | | |
3761 | 3761 | | |
3762 | | - | |
3763 | | - | |
3764 | | - | |
3765 | | - | |
3766 | | - | |
3767 | | - | |
3768 | | - | |
3769 | | - | |
3770 | | - | |
3771 | 3762 | | |
3772 | 3763 | | |
3773 | 3764 | | |
| |||
3777 | 3768 | | |
3778 | 3769 | | |
3779 | 3770 | | |
3780 | | - | |
| 3771 | + | |
3781 | 3772 | | |
3782 | 3773 | | |
| 3774 | + | |
3783 | 3775 | | |
3784 | 3776 | | |
3785 | 3777 | | |
3786 | 3778 | | |
3787 | | - | |
| 3779 | + | |
3788 | 3780 | | |
3789 | 3781 | | |
3790 | 3782 | | |
| |||
3795 | 3787 | | |
3796 | 3788 | | |
3797 | 3789 | | |
3798 | | - | |
3799 | | - | |
3800 | | - | |
3801 | | - | |
3802 | | - | |
3803 | | - | |
3804 | | - | |
3805 | | - | |
3806 | | - | |
3807 | 3790 | | |
3808 | 3791 | | |
3809 | 3792 | | |
| |||
3834 | 3817 | | |
3835 | 3818 | | |
3836 | 3819 | | |
3837 | | - | |
3838 | | - | |
3839 | | - | |
3840 | | - | |
3841 | | - | |
3842 | | - | |
3843 | | - | |
3844 | | - | |
3845 | | - | |
3846 | | - | |
3847 | | - | |
3848 | | - | |
3849 | | - | |
3850 | | - | |
3851 | | - | |
3852 | | - | |
3853 | | - | |
3854 | | - | |
3855 | | - | |
3856 | | - | |
3857 | | - | |
3858 | | - | |
3859 | | - | |
3860 | | - | |
3861 | | - | |
3862 | | - | |
3863 | | - | |
3864 | | - | |
3865 | | - | |
3866 | | - | |
3867 | | - | |
3868 | | - | |
3869 | | - | |
3870 | | - | |
3871 | | - | |
3872 | | - | |
3873 | | - | |
3874 | | - | |
3875 | | - | |
3876 | | - | |
3877 | | - | |
3878 | | - | |
3879 | | - | |
3880 | | - | |
3881 | | - | |
3882 | | - | |
3883 | | - | |
3884 | | - | |
3885 | | - | |
3886 | | - | |
3887 | | - | |
3888 | | - | |
3889 | | - | |
3890 | | - | |
3891 | 3820 | | |
3892 | 3821 | | |
3893 | 3822 | | |
| |||
6029 | 5958 | | |
6030 | 5959 | | |
6031 | 5960 | | |
6032 | | - | |
6033 | | - | |
6034 | | - | |
6035 | | - | |
6036 | | - | |
6037 | | - | |
6038 | | - | |
6039 | | - | |
6040 | | - | |
6041 | 5961 | | |
6042 | 5962 | | |
6043 | 5963 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
188 | 188 | | |
189 | 189 | | |
190 | 190 | | |
191 | | - | |
192 | | - | |
193 | | - | |
194 | | - | |
195 | | - | |
196 | | - | |
197 | | - | |
| 191 | + | |
198 | 192 | | |
199 | 193 | | |
200 | 194 | | |
| |||
233 | 227 | | |
234 | 228 | | |
235 | 229 | | |
236 | | - | |
237 | 230 | | |
238 | | - | |
239 | | - | |
240 | 231 | | |
241 | 232 | | |
242 | 233 | | |
| |||
This file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
0 commit comments