Skip to content

LiveTestWidgetsFlutterBinding gets confused by warm-up frames #132969

@goderbauer

Description

@goderbauer

LiveTestWidgetsFlutterBinding (used by package:integration_test and when flutter runing a package:flutter_test) aggressively controls when frames are allowed to render (to simulate the behavior of the AutomatedTestWidgetsFlutterBinding, which puts test authors fully in control of driving frames):

void handleBeginFrame(Duration? rawTimeStamp) {
assert(_doDrawThisFrame == null);
if (_expectingFrame ||
_expectingFrameToReassemble ||
(framePolicy == LiveTestWidgetsFlutterBindingFramePolicy.fullyLive) ||
(framePolicy == LiveTestWidgetsFlutterBindingFramePolicy.benchmarkLive) ||
(framePolicy == LiveTestWidgetsFlutterBindingFramePolicy.benchmark) ||
(framePolicy == LiveTestWidgetsFlutterBindingFramePolicy.fadePointers && _viewNeedsPaint)) {
_doDrawThisFrame = true;
super.handleBeginFrame(rawTimeStamp);
} else {
_doDrawThisFrame = false;
}
}

In a nutshell, a frame is only rendering if the test is doing an await tester.pump(), which will schedule a frame, set _expectingFrame to allow it to render, and then return a future that completes when the frame has rendered:

return TestAsyncUtils.guard<void>(() {
if (duration != null) {
Timer(duration, () {
_expectingFrame = true;
scheduleFrame();
});
} else {
_expectingFrame = true;
scheduleFrame();
}
_pendingFrame = Completer<void>();
return _pendingFrame!.future;
});
}

The problem is that warmup frames don't play nicely with this because they are asynchronously scheduled. If a warm-up frame is scheduled before the test executes await tester.pump() it is possible that the warm-up frame may complete the future returned by await (and not the frame that was actually scheduled by the pump). The actual frame scheduled by the pump is then blocked from rendering due to the tight control the binding has over frame rendering. This can create problems because the warm-up frame is not guaranteed to actually be rasterized and shown on screen by the engine. If the engine drops the warmup frame on the floor, the test is in an inconsistent state: It may incorrectly assume that something has rendered on screen (because the pump-Future completed) when in fact it hasn't (and it may never render if the test doesn't pump more frames). This can cause flakiness when taking screenshots among other problems.

Desired solution: Warm-up frames shouldn't complete the frame future returned by tester.pump().

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Issues that are less important to the Flutter projecta: tests"flutter test", flutter_test, or one of our testsframeworkflutter/packages/flutter repository. See also f: labels.team-frameworkOwned by Framework teamtriaged-frameworkTriaged by Framework team

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions