Skip to content

Conversation

@flar
Copy link
Contributor

@flar flar commented Jul 12, 2022

The E2E textfield benchmarks were not displaying a caret due to failing to pass along the benchmark driver properties. This problem will also affect all E2E benchmarks that use code to drive the benchmark (i.e. the textfield benchmarks had to click on the text field in order to get the caret to appear).

Also, adding a page delay to both the timeline and E2E version of these benchmarks to ensure that the page transition to the benchmark page finishes before the benchmarks start measuring.

@flutter-dashboard flutter-dashboard bot added a: text input Entering text in a text field or keyboard related problems c: contributor-productivity Team-specific productivity, code health, technical debt. labels Jul 12, 2022
@flar flar requested review from GaryQian and jason-simmons July 13, 2022 23:23
@flar flar added the autosubmit Merge PR when tree becomes green via auto submit App label Jul 14, 2022
@auto-submit auto-submit bot merged commit 4d0fd4f into flutter:master Jul 14, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 14, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/plugins that referenced this pull request Jul 14, 2022
@zanderso
Copy link
Member

SkiaPerf is recording a plethora of improvements and regressions on this change. Since it looks like we are changing how/what these benchmarks are measuring, my presumption is that this is expected, and doesn't actually reflect any actionable issue, but pasting the links here for confirmation:

improvements: https://flutter-flutter-perf.skia.org/e/?begin=1657752006&end=1657829288&keys=X8ba33df5c7778a24f0bed4ae067b1361&num_commits=50&request_type=1&xbaroffset=29858

regressions: https://flutter-flutter-perf.skia.org/e/?begin=1657752006&end=1657829288&keys=Xe5330c0ab6317b2859937225b4e7fedf&num_commits=50&request_type=1&xbaroffset=29858

@flar
Copy link
Contributor Author

flar commented Jul 15, 2022

For the improvements:

  • The textfield perf benchmark was specifically targeted by this PR so those improvements are definitely expected.
  • The animated placeholder is likely enjoying an improvement by honoring its pagedelay which was previously ignored, but which keeps it from measuring the impact of the noisy page transition that leads to it.

For the regressions:

  • The picture_cache_perf benchmark was not doing any scrolling so of course its values were all amazing before this fix. Now that the body closure is being passed along to the test function, it will be scrolling and measuring the performance of that scrolling - as originally intended - and the times will be worse, as expected.
  • But, there is a bug in the E2E version of the picture cache benchmark. The scroll deltas are inverted so rather than following the same pattern of the timeline version (3 x (scroll right twice then scroll left twice), it ends up testing (2x scroll-left-bump-animation then 3right and 2left). The bump animation is probably causing the big "worst" frame timing. Fixing that negative delta brings the results of the E2E benchmark in line with the timeline version. (See Fix the scroll delta negation in the E2E version of picture_cache_perf benchmark #107678 for a fix in flight)
  • The fullscreen text perf benchmark used to have its driver body ignored and so it never clicked to get a text caret, this PR fixed that so it is now measuring what it was intending to measure
  • The fullscreen perf benchmark did not have a delay to allow the page to settle (50ms is not enough) so I am adding one, hopefully that will bring its numbers back into line. (See add page delays to the fullscreen versions of the textfield perf benchmarks #107746 for a fix in flight)
  • The color_filter_and_fade E2D benchmark is now suffering from being measured for 10 seconds instead of 3. The 3 seconds only had fast frames where the animation had not been disrupted by restructuring the tree at opacity = 0/1. The new length of 10 seconds include one such disruption accounting for a long frame of 40ms.

@zanderso
Copy link
Member

Thanks for the detailed investigation!

camsim99 pushed a commit to camsim99/flutter that referenced this pull request Aug 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

a: text input Entering text in a text field or keyboard related problems autosubmit Merge PR when tree becomes green via auto submit App c: contributor-productivity Team-specific productivity, code health, technical debt.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants