Address frame policy benchmark flakes #155130
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Recently the microbenchmarks were flakey, but from an older bug. Turns out, `LiveTestWidgetsFlutterBindingFramePolicy` is defaulted to `fadePointers` with this fun note: > This can result in additional frames being pumped beyond those that the test itself requests, which can cause differences in behavior Both `text_intrinsic_bench` and `build_bench` use a similar pattern: * Load stocks app * Open the menu * Switch to `benchmark` frame policy What happens, rarely, is that `LiveTestWidgetsFlutterBinding.pumpBenchmark()` will call (async) `handleBeginFrame` and `handleDrawFrame`. `handleDrawFrame` juggles a tri-state boolean (null, false, true). This boolean is only reset to `null` when handleDrawFrame is called back to back, say, from an extra frame that was scheduled. 1. Switch tri-state boolean to an enum, its easier to read 2. remove asserts that compile away in benchmarks (`--profile`) 3. use `Error.throwWithStackTrace` to keep stack traces. Fixes flutter#150542 Fixes flutter#150543 - throw stack!
github-actions
bot
added
a: tests
Contributor
|
Generally pumpAndSettle is a red flag because it means the test is not going to be sensitive to changes in behaviour (e.g. if we start pumping more frames than expected, it'll just be silently accepted). In this case this isn't really a test so it's probably fine? It'd be good to know why we're getting these extra frames sometimes and fix that, though. I would only expect Also, can we switch to benchmark mode earlier maybe? Either way, LGTM. |
Hixie
reviewed
Sep 12, 2024
|
|
||
| if (caughtException != null) { | ||
| throw caughtException as Object; // ignore: only_throw_errors, rethrowing caught exception. | ||
| Error.throwWithStackTrace(caughtException as Object, stackTrace!); |
Contributor
There was a problem hiding this comment.
nice, is this api new? there's probably other places we could do this
Member
There was a problem hiding this comment.
It was new in Dart 2.16:
https://api.dart.dev/stable/3.4.4/dart-core/Error/throwWithStackTrace.html
It is indeed a nice feature!
Member
Author
There was a problem hiding this comment.
I was scratching my head looking at this saying "there's got to be another way" and then I found the bug you filed, @Hixie, wanting exactly this. The VM team has the suggestion there to use this new API.
Member
Author
There was a problem hiding this comment.
breadcrumbs: dart-lang/sdk#35494 -> dart-lang/sdk#30741 (comment)
jtmcdole
added
the
autosubmit
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Sep 13, 2024
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Sep 13, 2024
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Sep 13, 2024
auto-submit bot
pushed a commit
to flutter/packages
that referenced
this pull request
Sep 13, 2024
flutter/flutter@303f222...2d30fe4 2024-09-13 [email protected] Roll Packages from 91caa7a to 330581f (4 revisions) (flutter/flutter#155171) 2024-09-13 [email protected] Fix: Flicker when reorderable list doesn't change its position (flutter/flutter#151026) 2024-09-13 [email protected] Stop reading .packages from flutter_tools. (flutter/flutter#154912) 2024-09-13 [email protected] Roll Flutter Engine from 70109e3b40c0 to bef48e87f438 (1 revision) (flutter/flutter#155156) 2024-09-13 [email protected] Roll Flutter Engine from d917a15823f3 to 70109e3b40c0 (1 revision) (flutter/flutter#155151) 2024-09-13 [email protected] Roll Flutter Engine from 94696ed75dea to d917a15823f3 (1 revision) (flutter/flutter#155147) 2024-09-13 [email protected] Fix TextField content should be selected on desktop when gaining focus (flutter/flutter#154916) 2024-09-13 [email protected] Roll Flutter Engine from 04802b779045 to 94696ed75dea (1 revision) (flutter/flutter#155144) 2024-09-13 [email protected] Roll Flutter Engine from 3d8163f47919 to 04802b779045 (2 revisions) (flutter/flutter#155138) 2024-09-13 [email protected] Roll Flutter Engine from 4d5fea97e933 to 3d8163f47919 (1 revision) (flutter/flutter#155136) 2024-09-13 [email protected] Mark `_LayoutBuilderElement` as always clean (flutter/flutter#154694) 2024-09-13 [email protected] Roll Flutter Engine from 8609af642725 to 4d5fea97e933 (7 revisions) (flutter/flutter#155134) 2024-09-12 [email protected] Disable fuchsia in flutter_tools (flutter/flutter#155111) 2024-09-12 [email protected] Address frame policy benchmark flakes (flutter/flutter#155130) 2024-09-12 [email protected] Roll Flutter Engine from 48ddaf578fb0 to 8609af642725 (11 revisions) (flutter/flutter#155128) 2024-09-12 49699333+dependabot[bot]@users.noreply.github.com Bump peter-evans/create-pull-request from 7.0.1 to 7.0.2 (flutter/flutter#155126) 2024-09-12 [email protected] Prevent the keyboard from reshowing on iOS (flutter/flutter#154584) 2024-09-12 [email protected] fix(Linux): specify application id (flutter/flutter#154522) 2024-09-12 [email protected] update changelog on master (flutter/flutter#155109) 2024-09-12 [email protected] iOS: update provisioning profile for 2024-2025 cert (flutter/flutter#155101) 2024-09-12 [email protected] Roll Packages from 4c18648 to 91caa7a (2 revisions) (flutter/flutter#155103) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC [email protected],[email protected] on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Dec 11, 2024
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Dec 12, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Recently the microbenchmarks were flakey, but from an older bug. Turns out,
LiveTestWidgetsFlutterBindingFramePolicyis defaulted tofadePointerswith this fun note:Both
text_intrinsic_benchandbuild_benchuse a similar pattern:benchmarkframe policyWhat happens, rarely, is that
LiveTestWidgetsFlutterBinding.pumpBenchmark()will call (async)handleBeginFrameandhandleDrawFrame.handleDrawFramejuggles a tri-state boolean (null, false, true). This boolean is only reset tonullwhen handleDrawFrame is called back to back, say, from an extra frame that was scheduled.--profile)Error.throwWithStackTraceto keep stack traces.I've been running this test on device lab hardware for hundreds of runs and have not hit a failure yet.
Fixes #150542
Fixes #150543 - throw stack!