[google_maps_flutter] Some iOS integration tests failing in CI #139825
Description
We're seeing OOB persistent failures in a number of tests on iOS:
00:18 +8: testInitialCenterLocationAtCenter
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following TestFailure was thrown running a test:
Expected: <400>
Actual: <9223372036854775807>
When the exception was thrown, this was the stack:
#4 main.<anonymous closure> (file:///Volumes/Work/s/w/ir/x/w/packages/packages/google_maps_flutter/google_maps_flutter_ios/example/ios11/integration_test/google_maps_test.dart:340:5)
<asynchronous suspension>
#5 testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:168:15)
<asynchronous suspension>
#6 TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1017:5)
<asynchronous suspension>
<asynchronous suspension>
(elided one frame from package:stack_trace)
This was caught by the test expectation on the following line:
file:///Volumes/Work/s/w/ir/x/w/packages/packages/google_maps_flutter/google_maps_flutter_ios/example/ios11/integration_test/google_maps_test.dart line 340
The test description was:
testInitialCenterLocationAtCenter
[...]
00:18 +8 -1: testGetVisibleRegion
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following TestFailure was thrown running a test:
Expected: true
Actual: <false>
When the exception was thrown, this was the stack:
#4 main.<anonymous closure> (file:///Volumes/Work/s/w/ir/x/w/packages/packages/google_maps_flutter/google_maps_flutter_ios/example/ios11/integration_test/google_maps_test.dart:376:5)
<asynchronous suspension>
#5 testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:168:15)
<asynchronous suspension>
#6 TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1017:5)
<asynchronous suspension>
<asynchronous suspension>
(elided one frame from package:stack_trace)
This was caught by the test expectation on the following line:
file:///Volumes/Work/s/w/ir/x/w/packages/packages/google_maps_flutter/google_maps_flutter_ios/example/ios11/integration_test/google_maps_test.dart line 376
The test description was:
testGetVisibleRegion
════════════════════════════════════════════════════════════════════════════════════════════════════
00:18 +8 -2: testGetVisibleRegion [E]
Test failed. See exception logs above.
The test description was: testGetVisibleRegion
[...]
00:21 +17 -2: testGetZoomLevel
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following TestFailure was thrown running a test:
Expected: <7>
Actual: <5.0>
When the exception was thrown, this was the stack:
#4 main.<anonymous closure> (file:///Volumes/Work/s/w/ir/x/w/packages/packages/google_maps_flutter/google_maps_flutter_ios/example/ios11/integration_test/google_maps_test.dart:701:5)
<asynchronous suspension>
#5 testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:168:15)
<asynchronous suspension>
#6 TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1017:5)
<asynchronous suspension>
<asynchronous suspension>
(elided one frame from package:stack_trace)
This was caught by the test expectation on the following line:
file:///Volumes/Work/s/w/ir/x/w/packages/packages/google_maps_flutter/google_maps_flutter_ios/example/ios11/integration_test/google_maps_test.dart line 701
The test description was:
testGetZoomLevel
════════════════════════════════════════════════════════════════════════════════════════════════════
00:21 +17 -3: testGetZoomLevel [E]
Test failed. See exception logs above.
The test description was: testGetZoomLevel
[...]
00:22 +17 -3: testScreenCoordinate
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following TestFailure was thrown running a test:
Expected: ScreenCoordinate:<ScreenCoordinate(0, 0)>
Actual: ScreenCoordinate:<ScreenCoordinate(9223372036854775807, 9223372036854775807)>
When the exception was thrown, this was the stack:
#4 main.<anonymous closure> (file:///Volumes/Work/s/w/ir/x/w/packages/packages/google_maps_flutter/google_maps_flutter_ios/example/ios11/integration_test/google_maps_test.dart:735:5)
<asynchronous suspension>
#5 testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:168:15)
<asynchronous suspension>
#6 TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1017:5)
<asynchronous suspension>
<asynchronous suspension>
(elided one frame from package:stack_trace)
This was caught by the test expectation on the following line:
file:///Volumes/Work/s/w/ir/x/w/packages/packages/google_maps_flutter/google_maps_flutter_ios/example/ios11/integration_test/google_maps_test.dart line 735
The test description was:
testScreenCoordinate
════════════════════════════════════════════════════════════════════════════════════════════════════
00:22 +17 -4: testScreenCoordinate [E]
Test failed. See exception logs above.
The test description was: testScreenCoordinate
[...]
00:27 +20 -4: testTakeSnapshot - did not complete [E]
I can't reproduce them locally, so I don't have a good way to investigate right now. To re-open the tree, I'm going to disable these tests in google_maps_flutter_ios, as well as for iOS in google_maps_flutter (all with comments referencing this issue to make them findable).
Since it's OOB and I can't repro locally, my current speculation is either:
- a server change that broke us that hasn't rolled out everywhere, or
- the CI is being identified as malicious usage by the server.
The latter may be impacted by the fact that we aren't currently running with an API key in CI (although that can't explain it fully, because I don't have an API key set up locally). I do plan at some point to get the API key working in CI, but since it hasn't affected anything I haven't set up the plumbing, so I can't easily test whether the situation would be improved by having a key (e.g., if the problem is tripping a malicious usage detection, maybe a key would avoid that?)