Needing up to 20 taps for this to work... that seems like a bug we're hiding in this test.

Can you clarify why we need to tap this multiple times?

Is it possible we need something like waitForTappable instead?

/cc @jonahwilliams

Thanks for the feedback!

Is it possible we need something like waitForTappable instead?

• The waitForTappable is a good idea indeed. I did some local tests with this, but it didn't make any difference. See Linux_pixel_7pro integration_ui_keyboard_resize is 2.17% flaky #161300 (comment)
• I also tried some additional things - see Linux_pixel_7pro integration_ui_keyboard_resize is 2.17% flaky #161300 (comment)

Needing up to 20 taps for this to work... that seems like a bug we're hiding in this test.
Can you clarify why we need to tap this multiple times?

Can't explain it. I agree, this looks like a bug.

My first guess would be that it is a bug in FlutterDriver and/or how it interacts with Android's InputManager in the case of enableTextEntryEmulation: false.
Or possibly some issue in how FlutterView and InputManager interact, but in that case I would expect to see bug reports from production also. As opening an app and clicking on a TextField and expecting a keyboard to open is a very common use case.

As I propose in #161300 (comment), I think it would make sense to make a separate bug report for this specific bug description. Otherwise, it is somewhat obfuscated in #161300

The goal of making this PR is a temporary workaround to reduce the flakiness until the bug can be found and addressed. It seems it has been an issue for some time: #141301.
But if you feel this is premature and the workaround creates more confusion than it helps, then I propose we close this PR and try out some more strategies.

Speaking of that - as I can reproduce this locally quite easily, then if anyone has any suggestions I should try - let me know.

One observation: this also behaves slightly differently depending on the Android version and/or speed of the device. For me, the faster the device / newer the Android version, the more the setup of the test fails (for me locally).

My first guess would be that it is a bug in FlutterDriver and/or how it interacts with Android's InputManager in the case of enableTextEntryEmulation: false.

In driver mode, you mean. Yeah that is possible, I have not dug into the code here (yet).

Or possibly some issue in how FlutterView and InputManager interact, but in that case I would expect to see bug reports from production also. As opening an app and clicking on a TextField and expecting a keyboard to open is a very common use case.

Also possible. I am going to add @justinmc to this PR, and see if he has insight (works on text input).

The goal of making this PR is a temporary workaround to reduce the flakiness until the bug can be found and addressed.

It's just not clear to me this does resolve any flakiness, at least locally I can still fairly easily run into:

00:07 +0 -1: end-to-end test Ensure keyboard dismissal resizes the view to original size [E]

  Expected: true
    Actual: <false>

It is possible the polling changes you made make it a bit less likely to occur, but tapping multiple times doesn't change focus, and only changing focus is supposed to open the keyboard. I ran out of time today to look in more detail, but you could please try adding some logging (even locally) and tell how often a for-loop retry causes the test to pass where it would otherwise fail?

Something like:

int retries = 0;

// ...

if (double.parse(heightWithKeyboardShown) < double.parse(startHeight)) {
  if (retries > 0) {
    print('>>> $retries retries were used to make this PASS.');
  }
  heightTextDidShrink = true;
  break;
} else {
  retries++;
}

... and let me know if you see this hit significantly? I'll try to come back to this tomorrow.

Good idea!

I added the logging in a separate branch my fork

I ran 100 iterations, using 2 different Android devices and OS'es to execute the test from. I disabled automatic retries, so the stats is about actually running 100 test iterations (with no failures).

hyperfine -r 10 --style=full 'dart bin/test_runner.dart test -t integration_ui_keyboard_resize >> log.txt'

Results as follows.

MacBook Pro M1 + Pixel 8 Pro API 36

• 15x "No retries were used to make this PASS."
• 85x "1 retries were used to make this PASS."
• 0x "2 retries were used to make this PASS."

ThinkPad X1 Extreme Gen 4i Windows 11 Pro + Pixel 8 Pro API 36

• 94x "No retries were used to make this PASS."
• 6x "1 retries were used to make this PASS."
• 0x "2 retries were used to make this PASS."

MacBook Pro M1 + Samsung Galaxy Tab S3 API 28

• 0x "No retries were used to make this PASS."
• 88x "1 retries were used to make this PASS."
• 12x "2 retries were used to make this PASS."

ThinkPad X1 Extreme Gen 4i Windows 11 Pro + Samsung Galaxy Tab S3 API 28

• 0x "No retries were used to make this PASS."
• 99x "1 retries were used to make this PASS."
• 1x "2 were used to make this PASS."

Note that tests were about 15 seconds slower per test on average when executed from the ThinkPad vs the Macbook.

It is possible the polling changes you made make it a bit less likely to occur, but tapping multiple times doesn't change focus, and only changing focus is supposed to open the keyboard.

Good point!

Based on this video https://drive.google.com/file/d/1dQmIPs7mCVjxocCvWB9KSfsaSiB7_Tyh/view?usp=sharing of a failure I was able to capture it looks to me the TextFlied gets the focus, just InputManager refuses to open the keyboard (fails on 4th attempt at the end). As this log line warns:

W/InputMethodManager(19804): Ignoring showSoftInput() as view=io.flutter.embedding.android.FlutterView{16610da VFE...... .F....ID 0,0-1008,2244 #1} is not served.

This log comes from https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/core/java/android/view/inputmethod/InputMethodManager.java;drc=8fae81cc40d98877629b28a16fcc72b736423bdb;l=2385

Also tried to add a second TextField to the test app (with a separate FocusNode etc) and tap on the second TextField if opening the keyboard doesn't work with the first one - same results.
It fails with both. At both times, complains about the same instance of a FlutterView.

During today's call with John and Ergo we generated some more ideas to try out here to collect more info:

• Try an infinite loop to wait for the keyboard - does it pass then? Or never passes sometimes?
• Add an initial delay at the test - does it affect it? Will it pass better then?
• Any additional verbose info/logs we can activate (from FlutterDriver, etc) to better understand the process/communication between the threads there?

I'll run these also tomorrow or day after tomorrow at the latest

@matanlurey @justinmc
Comparing the logs from the infinite loop test.
I see the following pattern here - the keyboard can only be opened after the D/InsetsController(...): hide(ime(), fromIme=false) appears in the log.

...
<-Test starts->
.. end-to-end test Ensure keyboard dismissal resizes the view to original size ..
<- All attempts to open the keyboard fail in this period ->
D/InsetsController(...): hide(ime(), fromIme=false)
<- All attempts to open the keyboard succeed now ->
I/ImeTracker(...): com.yourcompany.integration_ui:...: onCancelled at PHASE_CLIENT_ALREADY_HIDDEN
I/ImeTracker(...): com.yourcompany.integration_ui:...: onRequestShow at ORIGIN_CLIENT reason SHOW_SOFT_INPUT fromUser false
D/InsetsController(12023): show(ime(), fromIme=false)
<- Keyboard opens somewhere here .. ->
D/InsetsController(12023): Setting requestedVisibleTypes to -1 (was -9)
D/InputConnectionAdaptor(12023): The input method toggled cursor monitoring on
...

Looking a the timing,

• In the tests with no retries or 1 retry the D/InsetsController(...): hide(ime(), fromIme=false) takes ~ 1 second to appear from start of the test
• In the test with 54 retries it takes 21 seconds to appear

The D/InsetsController(...): hide(ime(), fromIme=false) log seems to be coming from https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/core/java/android/view/InsetsController.java;l=1208

I re-ran the 2000/2000 test with the same hardware and infinite loop setup. The results are surprisingly similar.

One new learning - if Google Assistant / Gemini accidentally triggers on the device while testing, then the test is unable to proceed until the user manually dismisses the assistant sheet (this is the 3279 retries).
The 54x iteration is curious and strange because it matches the first test 1:1.

Results (MacBook Pro M1 + Pixel 8 Pro API 36) - SUCCEEDED at 2000/2000

 992x 1 retries were used to make this PASS.
   1x 3279 retries were used to make this PASS.
   1x 54 retries were used to make this PASS.
 174x No retries were used to make this PASS.

Sadly, I do not have the Pixel 7 Pro device. But I'll try with an older Pixel (Pixel 3) and with the Pixel 7 Pro emulator. As the 2000 iteration tests take about 10h to run it will take some time.

I have not yet had the time to look into more when the D/InsetsController(...): hide(ime(), fromIme=false) exactly triggers in the Android Activity lifecycle and why it affects our test in this way. Any hints here are welcome.

Unable to reproduce longer retry loops on Pixel 3 API 31:

1893x 1 retries were used to make this PASS.
 107x No retries were used to make this PASS.

Tried 2000 iterations with 0 fails on Pixel 7 Pro emulator API 35 - nothing interesting.

1610x 1 retries were used to make this PASS.
   1x 2 retries were used to make this PASS.
 389x No retries were used to make this PASS

Thank you for all the work here.