Skip to content

Commit e47e131

Browse files
authored
android_hardware_smoke_test: Improve reliability (flutter#191374)
Issue flutter#191348 shows that the detection and retry behavior introduced in PR flutter#190110 did not solve the issue. Previously, engine cache cleanup was placed solely in `@AfterClass tearDownClass()`. When a test run failed mid-suite on attempt 1, JUnit aborted before `@AfterClass` could run. When attempt 2 launched, `MainActivity` retrieved the corrupted engine from cache, guaranteeing that subsequent attempts failed with blank screenshots, which matches what we see in the run history. To fix this, we move cache cleanup to `MainActivity.evictEngineCache()`. `FlutterActivityTest.kt` invokes this cleanup during class teardown and whenever an `EglInitializationException` or `BlankScreenshotException` occurs, ensuring any retry starts with a clean engine instance. We also reduce the retry cap in `run_android_hardware_smoke_tests.dart` from 3 attempts to 2 to avoid 30-minute LUCI timeouts on broken emulators, which we saw some of in tryjobs. I think this should be an infrastructure failure. So if an unrecoverable EGL collapse occurs across both attempts, the runner logs an infrastructure failure and exits with code 2 (`INFRA_FAILURE`) We also saw some logs which suggest HWUI repeatedly attempted 10-bit color format negotiation (`101010-2`), which fails on SwiftShader CPU drivers. This could contribute to an increased rate of EGL errors, so `MainActivity.onCreate()` now explicitly locks the window pixel format to `PixelFormat.RGBA_8888`. ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [AI contribution guidelines] and understand my responsibilities, or I am not using AI tools. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [AI contribution guidelines]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#ai-contribution-guidelines [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
1 parent 247215b commit e47e131

3 files changed

Lines changed: 74 additions & 33 deletions

File tree

‎dev/bots/suite_runners/run_android_hardware_smoke_tests.dart‎

Lines changed: 48 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ Future<void> runAndroidHardwareSmokeTests({
5151
try {
5252
_setAndroidManifestBackend(androidManifestXml, androidManifestContents, backend);
5353

54-
// 1. Run driver tests to generate reference screenshots (with retry loop)
54+
// 1. Run driver tests to generate reference screenshots
5555
final bool success = await _runRetryLoop(testDir);
5656
if (!success) {
5757
return;
@@ -69,6 +69,9 @@ Future<void> runAndroidHardwareSmokeTests({
6969
'-s',
7070
], workingDirectory: androidDir);
7171
}
72+
} on _InfrastructureException catch (e) {
73+
foundError(<String>[e.message]);
74+
io.exitCode = 2;
7275
} finally {
7376
// Restore original contents.
7477
androidManifestXml.writeAsStringSync(androidManifestContents);
@@ -100,8 +103,7 @@ void _setAndroidManifestBackend(File file, String contents, ImpellerBackend back
100103
}
101104

102105
Future<bool> _runRetryLoop(String testDir) async {
103-
const maxAttempts = 3;
104-
var exitCode = 0;
106+
const maxAttempts = 2;
105107

106108
for (var attempt = 1; attempt <= maxAttempts; attempt++) {
107109
// Clear logcat buffer before running
@@ -119,47 +121,64 @@ Future<bool> _runRetryLoop(String testDir) async {
119121
driveArgs.add('--no-build');
120122
}
121123

122-
final Command command = await startCommand('flutter', driveArgs, workingDirectory: testDir);
123-
exitCode = await command.process.exitCode;
124-
if (exitCode == 0) {
124+
final CommandResult result = await runCommand('flutter', driveArgs, workingDirectory: testDir);
125+
if (result.exitCode == 0) {
125126
return true;
126127
}
127128

128129
io.stderr.writeln(
129-
'flutter drive failed with exit code $exitCode on attempt $attempt/$maxAttempts.',
130+
'flutter drive failed with exit code ${result.exitCode} on attempt $attempt/$maxAttempts.',
130131
);
131132

132-
// Inspect the process logcat on failure to detect if a transient EGL/graphics context
133-
// negotiation error occurred during startup, enabling a safe activity/process level retry.
134-
final bool hasEglWarning = await _checkForTransientEglFailure();
135-
if (!hasEglWarning) {
136-
// Non-retryable error: exit immediately and log specific failure
137-
foundError(<String>[
138-
'Android Hardware Smoke Tests driver run failed with exit code $exitCode and no transient EGL warning was found in logcat.',
139-
]);
140-
return false;
141-
}
142-
143-
// Retryable EGL warning: log progress and continue if attempts remain
144-
if (attempt < maxAttempts) {
133+
final driverOutput = '${result.flattenedStdout}\n${result.flattenedStderr}';
134+
135+
// Inspect logcat and driver output on failure to detect if an unrecoverable EGL
136+
// graphics pipeline collapse occurred (requires both logcat EGL error and blank image failure).
137+
final bool hasEglWarning = await _checkForTransientEglFailure(driverOutput);
138+
if (hasEglWarning) {
139+
if (attempt < maxAttempts) {
140+
io.stderr.writeln(
141+
'Attempt $attempt failed with EGL/graphics error. Retrying with clean engine cache...',
142+
);
143+
continue;
144+
}
145145
io.stderr.writeln(
146-
'attempt $attempt of $maxAttempts: detected retryable EGL initialization warning. Retrying...',
146+
'INFRASTRUCTURE FAILURE: EGL / graphics driver pipeline collapsed on emulator after $maxAttempts attempts.',
147+
);
148+
throw _InfrastructureException(
149+
'INFRASTRUCTURE FAILURE: EGL / graphics driver pipeline collapsed on emulator after $maxAttempts attempts.',
147150
);
148151
}
152+
153+
foundError(<String>[
154+
'Android Hardware Smoke Tests driver run failed with exit code ${result.exitCode}.',
155+
]);
156+
return false;
149157
}
150158

151-
// Loop finished: exhausted all attempts
152-
foundError(<String>[
153-
'Android Hardware Smoke Tests driver run failed to initialize EGL after $maxAttempts attempts.',
154-
]);
155-
return false;
159+
throw StateError('Unreachable');
160+
}
161+
162+
class _InfrastructureException implements Exception {
163+
_InfrastructureException(this.message);
164+
final String message;
165+
@override
166+
String toString() => message;
156167
}
157168

158-
Future<bool> _checkForTransientEglFailure() async {
169+
Future<bool> _checkForTransientEglFailure(String driverOutput) async {
159170
try {
160171
final String logcatOutput = await runAndGetStdout('adb', <String>['logcat', '-d']).join('\n');
161-
return logcatOutput.contains('Failed to choose config with EGL_SWAP_BEHAVIOR_PRESERVED') ||
162-
logcatOutput.contains('Failed to initialize 101010-2 format');
172+
final bool hasEglWarning =
173+
logcatOutput.contains('Failed to choose config with EGL_SWAP_BEHAVIOR_PRESERVED') ||
174+
logcatOutput.contains('EGL_NOT_INITIALIZED') ||
175+
logcatOutput.contains('EGL_BAD_CONFIG');
176+
final bool hasRenderingError =
177+
driverOutput.contains('blank/empty') ||
178+
driverOutput.contains('BlankScreenshotException') ||
179+
driverOutput.contains('EglInitializationException');
180+
181+
return hasEglWarning && hasRenderingError;
163182
} catch (e) {
164183
io.stderr.writeln('Warning: Failed to check logcat for EGL failure: $e');
165184
return false;

‎dev/integration_tests/android_hardware_smoke_test/android/app/src/androidTest/java/com/example/android_hardware_smoke_test/FlutterActivityTest.kt‎

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,6 @@ import androidx.lifecycle.Lifecycle
1313
import androidx.test.ext.junit.rules.ActivityScenarioRule
1414
import androidx.test.ext.junit.runners.AndroidJUnit4
1515
import androidx.test.platform.app.InstrumentationRegistry
16-
import io.flutter.embedding.engine.FlutterEngineCache
1716
import org.json.JSONObject
1817
import org.junit.AfterClass
1918
import org.junit.Assert.assertEquals
@@ -45,15 +44,16 @@ class FlutterActivityTest {
4544
@AfterClass
4645
fun tearDownClass() {
4746
InstrumentationRegistry.getInstrumentation().runOnMainSync {
48-
val engine = FlutterEngineCache.getInstance().get(MainActivity.CACHED_ENGINE_KEY)
49-
engine?.destroy()
50-
FlutterEngineCache.getInstance().remove(MainActivity.CACHED_ENGINE_KEY)
47+
MainActivity.evictEngineCache()
5148
}
5249
}
5350

5451
private fun verifyNoGraphicsPipelineErrors(marker: String) {
5552
val errors = getGraphicsPipelineErrors(marker)
5653
if (errors.isNotEmpty()) {
54+
InstrumentationRegistry.getInstrumentation().runOnMainSync {
55+
MainActivity.evictEngineCache()
56+
}
5757
throw EglInitializationException(
5858
"Graphics pipeline/EGL failure detected in process logcat:\n${errors.joinToString("\n")}"
5959
)
@@ -319,6 +319,11 @@ class FlutterActivityTest {
319319
verifyNoGraphicsPipelineErrors(marker)
320320

321321
if (cropped == null) {
322+
// Evict engine cache if a screenshot is blank but no logcat EGL error was detected,
323+
// ensuring any subsequent test run attempt starts with a clean engine instance.
324+
InstrumentationRegistry.getInstrumentation().runOnMainSync {
325+
MainActivity.evictEngineCache()
326+
}
322327
throw BlankScreenshotException(
323328
"Captured screenshot is blank/empty after $maxAttempts attempts."
324329
)

‎dev/integration_tests/android_hardware_smoke_test/android/app/src/main/kotlin/com/example/android_hardware_smoke_test/MainActivity.kt‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,9 @@ package com.example.android_hardware_smoke_test
44

55
import android.content.Context
66
import android.content.pm.PackageManager
7+
import android.graphics.PixelFormat
78
import android.os.Build
9+
import android.os.Bundle
810
import android.util.Log
911
import io.flutter.embedding.android.FlutterActivity
1012
import io.flutter.embedding.engine.FlutterEngine
@@ -26,6 +28,21 @@ class MainActivity : FlutterActivity() {
2628

2729
// Tracks the active activity to prevent transition race conditions on the cached engine.
2830
private var activeActivity: WeakReference<MainActivity>? = null
31+
32+
// Destroys and removes the cached FlutterEngine and resets lastConfiguredEngine.
33+
fun evictEngineCache() {
34+
val engine = FlutterEngineCache.getInstance().get(CACHED_ENGINE_KEY)
35+
engine?.destroy()
36+
FlutterEngineCache.getInstance().remove(CACHED_ENGINE_KEY)
37+
lastConfiguredEngine = null
38+
}
39+
}
40+
41+
override fun onCreate(savedInstanceState: Bundle?) {
42+
// Lock window pixel format to 8-bit sRGB before FlutterActivity initialization
43+
// to prevent HWUI 10-bit format negotiation errors on SwiftShader drivers.
44+
window.setFormat(PixelFormat.RGBA_8888)
45+
super.onCreate(savedInstanceState)
2946
}
3047

3148
// Accessed by FlutterActivityTest to send orchestration messages.

0 commit comments

Comments
 (0)