Skip to content

Commit cf854df

Browse files
authored
[devicelab] Fix memory benchmark tap synchronization and swipe flakes (flutter#194457)
Fixes synchronization and timeout flakes in Devicelab memory benchmarks: - Replaces the `Future.any` race in `MemoryTest.tapNotification()` (which fired `adb shell input tap 100 100` every 100ms and could leave an unawaited tap in flight after `TAPPED` was received, causing stray taps or duplicate `Completer.complete()` throws in `flutter_gallery__memory_nav` and `complex_layout_scroll_perf__memory`) with a sequential retry loop (up to 10 attempts, 1s timeout per tap). - Guards `Completer.complete()` in `MemoryTest.handleLogcatLine()`, `ReportedDurationTest.run()`, `memory_nav.dart`, and `scroll_perf.dart` against duplicate benchmark markers. - Updates `fast_scroll_heavy_gridview__memory` and `fast_scroll_large_images__memory` to drive scrolling deterministically in-process with `LiveWidgetController(WidgetsBinding.instance).fling(...)` + `endOfAnimation()` and `==== MEMORY BENCHMARK ==== DONE ====` via the standard `MemoryTest.useMemory()` flow instead of blind `adb shell input swipe` coordinates and fixed delays. - Adds bounded 60-second timeouts (`MemoryTest.nextMessageTimeout` and `ReportedDurationTest.durationTimeout`) to `READY`, `DONE`, `AppLifecycleState.paused`, and `durationCompleter.future` logcat waits so missed log lines fail fast instead of hanging the LUCI builder for 30 minutes. Fixes flutter#149901 ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [AI contribution guidelines] and understand my responsibilities, or I am not using AI tools. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant in-code documentation (doc comments with `///`). - [x] If this PR introduces a new feature or capability, I created and linked a website documentation issue or PR in [flutter/website] (or verified none is needed). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [AI contribution guidelines]: https://github.com/flutter/flutter/blob/main/CONTRIBUTING.ai.md [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [flutter/website]: https://github.com/flutter/website [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
1 parent 1fc9dfa commit cf854df

9 files changed

Lines changed: 202 additions & 71 deletions

File tree

‎dev/benchmarks/complex_layout/test_memory/scroll_perf.dart‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,8 +24,10 @@ Future<void> main() async {
2424
runApp(
2525
GestureDetector(
2626
onTap: () {
27-
debugPrint('==== MEMORY BENCHMARK ==== TAPPED ====');
28-
ready.complete();
27+
if (!ready.isCompleted) {
28+
debugPrint('==== MEMORY BENCHMARK ==== TAPPED ====');
29+
ready.complete();
30+
}
2931
},
3032
behavior: HitTestBehavior.opaque,
3133
child: const IgnorePointer(child: ComplexLayoutApp()),

‎dev/benchmarks/macrobenchmarks/test_memory/heavy_gridview.dart‎

Lines changed: 48 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,21 +2,67 @@
22
// Use of this source code is governed by a BSD-style license that can be
33
// found in the LICENSE file.
44

5+
import 'dart:async';
6+
57
import 'package:flutter/material.dart';
68
import 'package:flutter/scheduler.dart';
79
import 'package:flutter_test/flutter_test.dart';
810
import 'package:macrobenchmarks/common.dart';
911
import 'package:macrobenchmarks/main.dart';
1012

13+
const Duration _initialSettleDelay = Duration(milliseconds: 50);
14+
const Duration _tapSyncDelay = Duration(milliseconds: 200);
15+
const Offset _flingOffset = Offset(0.0, -1000.0);
16+
const double _flingSpeed = 10000.0;
17+
const int _flingIterations = 3;
18+
1119
Future<void> endOfAnimation() async {
1220
do {
1321
await SchedulerBinding.instance.endOfFrame;
1422
} while (SchedulerBinding.instance.hasScheduledFrame);
1523
}
1624

1725
Future<void> main() async {
18-
runApp(const MacrobenchmarksApp(initialRoute: kHeavyGridViewRouteName));
26+
final ready = Completer<void>();
27+
runApp(
28+
GestureDetector(
29+
onTap: () {
30+
if (!ready.isCompleted) {
31+
debugPrint('==== MEMORY BENCHMARK ==== TAPPED ====');
32+
ready.complete();
33+
}
34+
},
35+
behavior: HitTestBehavior.opaque,
36+
child: const IgnorePointer(child: MacrobenchmarksApp(initialRoute: kHeavyGridViewRouteName)),
37+
),
38+
);
1939
await endOfAnimation();
20-
await Future<void>.delayed(const Duration(milliseconds: 50));
40+
await Future<void>.delayed(_initialSettleDelay);
2141
debugPrint('==== MEMORY BENCHMARK ==== READY ====');
42+
43+
await ready.future;
44+
debugPrint('Continuing...');
45+
46+
// Wait out any errant taps due to synchronization.
47+
await Future<void>.delayed(_tapSyncDelay);
48+
49+
// Remove onTap handler, enable pointer events for app.
50+
runApp(
51+
GestureDetector(
52+
child: const IgnorePointer(
53+
ignoring: false,
54+
child: MacrobenchmarksApp(initialRoute: kHeavyGridViewRouteName),
55+
),
56+
),
57+
);
58+
await endOfAnimation();
59+
60+
final WidgetController controller = LiveWidgetController(WidgetsBinding.instance);
61+
for (var iteration = 0; iteration < _flingIterations; iteration += 1) {
62+
debugPrint('Flinging... $iteration/$_flingIterations');
63+
await controller.fling(find.byType(Scrollable), _flingOffset, _flingSpeed);
64+
await endOfAnimation();
65+
}
66+
67+
debugPrint('==== MEMORY BENCHMARK ==== DONE ====');
2268
}

‎dev/benchmarks/macrobenchmarks/test_memory/large_images.dart‎

Lines changed: 45 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,21 +2,64 @@
22
// Use of this source code is governed by a BSD-style license that can be
33
// found in the LICENSE file.
44

5+
import 'dart:async';
6+
57
import 'package:flutter/material.dart';
68
import 'package:flutter/scheduler.dart';
79
import 'package:flutter_test/flutter_test.dart';
810
import 'package:macrobenchmarks/common.dart';
911
import 'package:macrobenchmarks/main.dart';
1012

13+
const Duration _initialSettleDelay = Duration(milliseconds: 50);
14+
const Duration _tapSyncDelay = Duration(milliseconds: 200);
15+
const Offset _flingOffset = Offset(0.0, -1000.0);
16+
const double _flingSpeed = 10000.0;
17+
1118
Future<void> endOfAnimation() async {
1219
do {
1320
await SchedulerBinding.instance.endOfFrame;
1421
} while (SchedulerBinding.instance.hasScheduledFrame);
1522
}
1623

1724
Future<void> main() async {
18-
runApp(const MacrobenchmarksApp(initialRoute: kLargeImagesRouteName));
25+
final ready = Completer<void>();
26+
runApp(
27+
GestureDetector(
28+
onTap: () {
29+
if (!ready.isCompleted) {
30+
debugPrint('==== MEMORY BENCHMARK ==== TAPPED ====');
31+
ready.complete();
32+
}
33+
},
34+
behavior: HitTestBehavior.opaque,
35+
child: const IgnorePointer(child: MacrobenchmarksApp(initialRoute: kLargeImagesRouteName)),
36+
),
37+
);
1938
await endOfAnimation();
20-
await Future<void>.delayed(const Duration(milliseconds: 50));
39+
await Future<void>.delayed(_initialSettleDelay);
2140
debugPrint('==== MEMORY BENCHMARK ==== READY ====');
41+
42+
await ready.future;
43+
debugPrint('Continuing...');
44+
45+
// Wait out any errant taps due to synchronization.
46+
await Future<void>.delayed(_tapSyncDelay);
47+
48+
// Remove onTap handler, enable pointer events for app.
49+
runApp(
50+
GestureDetector(
51+
child: const IgnorePointer(
52+
ignoring: false,
53+
child: MacrobenchmarksApp(initialRoute: kLargeImagesRouteName),
54+
),
55+
),
56+
);
57+
await endOfAnimation();
58+
59+
final WidgetController controller = LiveWidgetController(WidgetsBinding.instance);
60+
debugPrint('Flinging...');
61+
await controller.fling(find.byType(Scrollable), _flingOffset, _flingSpeed);
62+
await endOfAnimation();
63+
64+
debugPrint('==== MEMORY BENCHMARK ==== DONE ====');
2265
}

‎dev/devicelab/bin/tasks/fast_scroll_heavy_gridview__memory.dart‎

Lines changed: 1 addition & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -15,26 +15,11 @@ class FastScrollHeavyGridViewMemoryTest extends MemoryTest {
1515
'${flutterDirectory.path}/dev/benchmarks/macrobenchmarks',
1616
'test_memory/heavy_gridview.dart',
1717
kPackageName,
18+
requiresTapToStart: true,
1819
);
1920

20-
@override
21-
AndroidDevice? get device => super.device as AndroidDevice?;
22-
2321
@override
2422
int get iterationCount => 5;
25-
26-
@override
27-
Future<void> useMemory() async {
28-
await launchApp();
29-
await recordStart();
30-
await device!.shellExec('input', <String>['swipe', '50 1500 50 50 50']);
31-
await Future<void>.delayed(const Duration(milliseconds: 1500));
32-
await device!.shellExec('input', <String>['swipe', '50 1500 50 50 50']);
33-
await Future<void>.delayed(const Duration(milliseconds: 1500));
34-
await device!.shellExec('input', <String>['swipe', '50 1500 50 50 50']);
35-
await Future<void>.delayed(const Duration(milliseconds: 1500));
36-
await recordEnd();
37-
}
3823
}
3924

4025
Future<void> main() async {

‎dev/devicelab/bin/tasks/fast_scroll_large_images__memory.dart‎

Lines changed: 1 addition & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -15,22 +15,11 @@ class FastScrollLargeImagesMemoryTest extends MemoryTest {
1515
'${flutterDirectory.path}/dev/benchmarks/macrobenchmarks',
1616
'test_memory/large_images.dart',
1717
kPackageName,
18+
requiresTapToStart: true,
1819
);
1920

20-
@override
21-
AndroidDevice? get device => super.device as AndroidDevice?;
22-
2321
@override
2422
int get iterationCount => 5;
25-
26-
@override
27-
Future<void> useMemory() async {
28-
await launchApp();
29-
await recordStart();
30-
await device!.shellExec('input', <String>['swipe', '0 1500 0 0 50']);
31-
await Future<void>.delayed(const Duration(milliseconds: 15000));
32-
await recordEnd();
33-
}
3423
}
3524

3625
Future<void> main() async {

‎dev/devicelab/bin/tasks/flutter_gallery__back_button_memory.dart‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ class BackButtonMemoryTest extends MemoryTest {
3737
// Push back button, wait for it to be seen by the Flutter app.
3838
prepareForNextMessage('AppLifecycleState.paused');
3939
await device!.shellExec('input', <String>['keyevent', 'KEYCODE_BACK']);
40-
await receivedNextMessage;
40+
await receivedNextMessage?.timeout(MemoryTest.nextMessageTimeout);
4141

4242
// Give Android time to settle (e.g. run GCs) after closing the app.
4343
await Future<void>.delayed(const Duration(milliseconds: 100));
@@ -53,7 +53,7 @@ class BackButtonMemoryTest extends MemoryTest {
5353
if (output.contains('Error')) {
5454
fail('unable to launch activity');
5555
}
56-
await receivedNextMessage;
56+
await receivedNextMessage?.timeout(MemoryTest.nextMessageTimeout);
5757

5858
// Wait for the Flutter app to settle (e.g. run GCs).
5959
await Future<void>.delayed(const Duration(milliseconds: 100));

‎dev/devicelab/lib/tasks/perf_tests.dart‎

Lines changed: 38 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -2167,6 +2167,11 @@ class CompileTest {
21672167
class MemoryTest {
21682168
MemoryTest(this.project, this.test, this.package, {this.requiresTapToStart = false});
21692169

2170+
static const Duration nextMessageTimeout = Duration(seconds: 60);
2171+
static const Duration _tapRetryTimeout = Duration(seconds: 1);
2172+
static const int _maxTapIterations = 10;
2173+
static const int _tapCoordinate = 100;
2174+
21702175
final String project;
21712176
final String test;
21722177
final String package;
@@ -2185,6 +2190,18 @@ class MemoryTest {
21852190
_receivedNextMessage = Completer<void>();
21862191
}
21872192

2193+
/// Completes [receivedNextMessage] if [line] contains the expected
2194+
/// memory benchmark marker for the last [prepareForNextMessage] call.
2195+
@visibleForTesting
2196+
void handleLogcatLine(String line) {
2197+
if (_nextMessage != null &&
2198+
line.contains('==== MEMORY BENCHMARK ==== $_nextMessage ====') &&
2199+
!(_receivedNextMessage?.isCompleted ?? true)) {
2200+
_nextMessage = null;
2201+
_receivedNextMessage?.complete();
2202+
}
2203+
}
2204+
21882205
int get iterationCount => 10;
21892206

21902207
Device? get device => _device;
@@ -2199,11 +2216,7 @@ class MemoryTest {
21992216
await device!.unlock();
22002217
await flutter('packages', options: <String>['get']);
22012218

2202-
final StreamSubscription<String> adb = device!.logcat.listen((String data) {
2203-
if (data.contains('==== MEMORY BENCHMARK ==== $_nextMessage ====')) {
2204-
_receivedNextMessage?.complete();
2205-
}
2206-
});
2219+
final StreamSubscription<String> adb = device!.logcat.listen(handleLogcatLine);
22072220

22082221
for (var iteration = 0; iteration < iterationCount; iteration += 1) {
22092222
print('running memory test iteration $iteration...');
@@ -2250,8 +2263,8 @@ class MemoryTest {
22502263
'run',
22512264
options: <String>['--verbose', '--release', '--no-resident', '-d', device!.deviceId, test],
22522265
);
2253-
print('awaiting "ready" message...');
2254-
await receivedNextMessage;
2266+
print('awaiting "ready" message... (timeout: $nextMessageTimeout)');
2267+
await receivedNextMessage?.timeout(nextMessageTimeout);
22552268
}
22562269

22572270
/// Taps the application and looks for acknowledgement.
@@ -2261,29 +2274,19 @@ class MemoryTest {
22612274
// Keep "tapping" the device till it responds with the string we expect,
22622275
// or throw an error instead of tying up the infrastructure for 30 minutes.
22632276
prepareForNextMessage('TAPPED');
2264-
var tapped = false;
2265-
var tapCount = 0;
2266-
await Future.any(<Future<void>>[
2267-
() async {
2268-
while (true) {
2269-
if (tapped) {
2270-
break;
2271-
}
2272-
tapCount += 1;
2273-
print('tapping device... [$tapCount]');
2274-
await device!.tap(100, 100);
2275-
await Future<void>.delayed(const Duration(milliseconds: 100));
2276-
}
2277-
}(),
2278-
() async {
2279-
print('awaiting "tapped" message... (timeout: 10 seconds)');
2280-
try {
2281-
await receivedNextMessage?.timeout(const Duration(seconds: 10));
2282-
} finally {
2283-
tapped = true;
2277+
for (var tapCount = 1; tapCount <= _maxTapIterations; tapCount += 1) {
2278+
print('tapping device... [$tapCount]');
2279+
await device!.tap(_tapCoordinate, _tapCoordinate);
2280+
print('awaiting "tapped" message... (timeout: $_tapRetryTimeout)');
2281+
try {
2282+
await receivedNextMessage?.timeout(_tapRetryTimeout);
2283+
return;
2284+
} on TimeoutException {
2285+
if (tapCount == _maxTapIterations) {
2286+
rethrow;
22842287
}
2285-
}(),
2286-
]);
2288+
}
2289+
}
22872290
}
22882291

22892292
/// To change the behavior of the test, override this.
@@ -2300,8 +2303,8 @@ class MemoryTest {
23002303
}
23012304

23022305
prepareForNextMessage('DONE');
2303-
print('awaiting "done" message...');
2304-
await receivedNextMessage;
2306+
print('awaiting "done" message... (timeout: $nextMessageTimeout)');
2307+
await receivedNextMessage?.timeout(nextMessageTimeout);
23052308

23062309
await recordEnd();
23072310
}
@@ -2398,6 +2401,8 @@ String _reportedDurationTestToString(ReportedDurationTestFlavor flavor) {
23982401
class ReportedDurationTest {
23992402
ReportedDurationTest(this.flavor, this.project, this.test, this.package, this.durationPattern);
24002403

2404+
static const Duration durationTimeout = Duration(seconds: 60);
2405+
24012406
final ReportedDurationTestFlavor flavor;
24022407
final String project;
24032408
final String test;
@@ -2421,7 +2426,7 @@ class ReportedDurationTest {
24212426
await flutter('packages', options: <String>['get']);
24222427

24232428
final StreamSubscription<String> adb = device!.logcat.listen((String data) {
2424-
if (durationPattern.hasMatch(data)) {
2429+
if (durationPattern.hasMatch(data) && !durationCompleter.isCompleted) {
24252430
durationCompleter.complete(int.parse(durationPattern.firstMatch(data)!.group(1)!));
24262431
}
24272432
});
@@ -2439,7 +2444,7 @@ class ReportedDurationTest {
24392444
],
24402445
);
24412446

2442-
final int duration = await durationCompleter.future;
2447+
final int duration = await durationCompleter.future.timeout(durationTimeout);
24432448
print('terminating...');
24442449
await device!.stop(package);
24452450
await adb.cancel();

0 commit comments

Comments
 (0)