Repository navigation
Integration test runs twice on web platform #102250
Description
Steps to Reproduce
- Execute
flutter drive --driver=test_driver/integration_test.dart --target=integration_test/example_test.dart -d chromeon the code sample - Observe that the test fails as 2 documents have been inserted to Firestore
- If ran on other platform (iOS) the same behavior is not observed
Expected results:
When a single test is ran that inserts a single document a single document should be inserted
00:00 +0: (setUpAll)
00:00 +1: Single document created
Pre-existing documents: 0
00:01 +2: (tearDownAll)
00:01 +3: All tests passed!
Actual results:
When a single test is ran two documents are inserted, I think that the test is running twice
00:00 +0: (setUpAll)
00:00 +1: Single document created
Pre-existing documents: 0
00:01 +2: (tearDownAll)
00:01 +3: All tests passed!
Failure Details:
Failure in method: Single document created
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
The following TestFailure was thrown running a test:
Expected: <1>
Actual: <2>
Code sample
void main() {
IntegrationTestWidgetsFlutterBinding.ensureInitialized();
final now = DateTime.now();
const String collectionName = 'examples';
String currentMinute = DateFormat.yMd().add_jm().format(now);
documentsInsertedThisMinute() async {
var collection = FirebaseFirestore.instance.collection(collectionName);
var snapshots =
await collection.where('foobar', isEqualTo: currentMinute).get();
return snapshots.size;
}
setUpAll(() async {
await Firebase.initializeApp(
options: DefaultFirebaseOptions.currentPlatform,
);
});
testWidgets(
'Single document created',
(WidgetTester tester) async {
int preExistingDocuments = await documentsInsertedThisMinute();
print('Pre-existing documents: $preExistingDocuments');
await tester.pumpWidget(const MyApp());
await FirebaseFirestore.instance
.collection(collectionName)
.doc()
.set({'foobar': currentMinute});
int documentsAfterSingleInsert = await documentsInsertedThisMinute();
expect(documentsAfterSingleInsert, 1);
},
);
}
Logs
Gist of flutter drive --verbose
Analyzing integration_tests_duplicated...
info • Avoid `print` calls in production code • integration_test/example_test.dart:32:7 • avoid_print
1 issue found. (ran in 2.2s)
[✓] Flutter (Channel stable, 2.10.4, on macOS 12.2.1 21D62 darwin-arm, locale en-US)
• Flutter version 2.10.4 at /Users/garrett/development/flutter
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision c860cba910 (4 weeks ago), 2022-03-25 00:23:12 -0500
• Engine revision 57d3bac3dd
• Dart version 2.16.2
• DevTools version 2.9.2
[✓] Android toolchain - develop for Android devices (Android SDK version 32.1.0-rc1)
• Android SDK at /Users/garrett/Library/Android/sdk
• Platform android-32, build-tools 32.1.0-rc1
• Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7772763)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS (Xcode 13.2.1)
• Xcode at /Applications/Xcode.app/Contents/Developer
• CocoaPods version 1.11.3
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 2021.1)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7772763)
[✓] VS Code (version 1.66.2)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.38.1
[✓] Connected device (3 available)
• Garrett’s iPad (mobile) • 00000000-0000000000000000 • ios • iOS 15.3.1 19D52
• macOS (desktop) • macos • darwin-arm64 • macOS 12.2.1 21D62 darwin-arm
• Chrome (web) • chrome • web-javascript • Google Chrome 100.0.4896.127
[✓] HTTP Host Availability
• All required HTTP hosts are available
• No issues found!
Reacted by Donald Taylor, garrettApproachableGeek, Ivanro Lagazo, Hue, Zachary Bohn and emmapejko
Activity
darshankawar commentedon Apr 21, 2022
@garrettApproachableGeek
Thanks for the detailed report. I think this is the case of two chrome instances being launched in the background and hence you might be seeing the behavior.
For example, when running with
-d chromeflag, one chrome instance is opened and notice that it is not labelled withchrome is controlled by automated test software.Also, if you run with
-d chrome \ --no-headlessthen the chrome instance will showchrome is controlled by automated test software.In both above cases, the chrome instances are overlapped with each other and if you try to drag the front chrome instance away, you might see another instance.
Can you confirm above behavior ?
garrettApproachableGeek commentedon Apr 21, 2022
When running with --no-headless I do see that there are two chrome instances and one of them says
chrome is controlled by automated test softwareHowever when running without the --no-headless flag I only see one chrome instance, but I assume there is a headless instance running that is not visible?
darshankawar commentedon Apr 22, 2022
Yes and that leads me to this open issue #67090, so I suggest you to follow-up in it for further updates.
Closing this as a duplicate, since root-cause is same as mentioned in linked issue.
github-actions commentedon May 6, 2022
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of
flutter doctor -vand a minimal reproduction of the issue.