-
Notifications
You must be signed in to change notification settings - Fork 29.7k
Do not hang on test failures of tests within flutter_tools
#141821
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Do not hang on test failures of tests within flutter_tools
#141821
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, pending unit test results of course
|
What role is runZoneGuarded playing here? |
AFAICT, For example: // Will catch errors _returned_ by body, but not as a side-effect of body.
try {
await body();
} catch (e) {
print(e);
} |
right. To put it another way, as far as I understand, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this redundant with the one on line 169?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, these are errors caught within the executing future versus outside 😓
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't match my understanding - with the rethrow here I expect that error will also be passed to the onError callback argument, and _printBufferedErrors(context); will be called twice in that case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@natebosch Thanks! Is your suggestion cutting out this catch block or something else?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I think you can remove this catch and keep the finally to complete the completer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you sir!
|
So this is super confusing and easy to get wrong, but it does actually make sense :). The entire purpose of this API is to swallow errors, it does not allow them to escape the Zone. This includes async errors. So, if there is an error attached to a Future which escapes the Zone (such as being returned from the |
|
Fwiw I thought we added a different API which does essentially what your fix is, and has a return type of |
|
@jakemac53 fwiw our general discontent around this is largely that Zones are basically entirely undocumented, so it's all guesswork when we try to use them. For example, dart-lang/sdk#26455, dart-lang/sdk#54177, dart-lang/sdk#17667, dart-lang/sdk#44669, dart-lang/sdk#26454, ... |
|
Sure, I have felt that pain many times before :). Even where there are docs I find them incredibly difficult to understand. I think the number of people who would self-proclaim to understand zones fully is 2, possibly just 1. I am not one of them :). But, for this particular issue, I hope the simplified explanation I provided above helps illuminate what is actually happening here? |
|
I'm trying to encourage you to explain zones by submitting CLs to improve the API docs rather than just commenting on PRs. :-) |
…t-hang-on-failure
9392ce7 to
b3eabb6
Compare
flutter test <x> would have a test failure, do not hangflutter_tools
I think you might be conflating another API. We don't have any API that expects a value returned from a import 'dart:async';
void main() async {
final value = await runZonedGuarded(() {
Future.error('Some error'); // Not awaited - unhandled async error.
return Future.delayed(const Duration(milliseconds: 500), () {
return 'Some value';
});
}, (e, st) {
print('Unhandled async error: $e\n$st');
});
print('Returned value: $value');
} |
|
@lrhn - WDYT about adding an API in /// Runs [callback] in a zone and returns the result or the first
/// unhandled error that occurs in the zone.
///
/// If an unhandled asynchronous error occurs before the Future
/// returned from [callback] has completed, the callback result
/// will be ignored (if it ever completes), and the returned Future
/// will complete with the same error.
///
/// If there are multiple unhandled asynchronous errors, or if the
/// Future returned by [callback] has an error after an unhandled
/// asynchronous error, all but the first occurring error is ignored.
///
/// This API may be useful to avoid Futures that never complete
/// due to an error that cannot escape it's error zone.
Future<T> Result.runZonedSmuggleErrors(Future<T> Function() callback);Maybe we can also have a callback to handle those subsequent errors instead of strictly ignoring them. |
|
Yeah I think that is really the API most people are expecting here. |
flutter/flutter@3ee8ff2...5b673c2 2024-01-23 [email protected] Roll Flutter Engine from fd0335a910b8 to b229878c57f5 (1 revision) (flutter/flutter#142051) 2024-01-23 [email protected] Remove unused clipBehavior from OverflowBar (flutter/flutter#141976) 2024-01-23 [email protected] Roll Packages from e4cbf23 to 841fe90 (7 revisions) (flutter/flutter#142047) 2024-01-23 [email protected] Roll Flutter Engine from d2855da628da to fd0335a910b8 (1 revision) (flutter/flutter#142046) 2024-01-23 [email protected] Add Share button to the SelectableRegion toolbar on Android (flutter/flutter#141447) 2024-01-23 [email protected] Relax the warning of unavailable tokens in `gen_defaults` when a default value is provided (flutter/flutter#140872) 2024-01-23 [email protected] Roll Flutter Engine from 37f68f6fc7fc to d2855da628da (1 revision) (flutter/flutter#142033) 2024-01-23 [email protected] Roll Flutter Engine from 9e582c9032e5 to 37f68f6fc7fc (1 revision) (flutter/flutter#142027) 2024-01-23 [email protected] Roll Flutter Engine from df6b15d35703 to 9e582c9032e5 (1 revision) (flutter/flutter#142026) 2024-01-23 [email protected] Roll Flutter Engine from d3dbd4225e08 to df6b15d35703 (6 revisions) (flutter/flutter#142023) 2024-01-23 [email protected] Add a comment about how to test flutter_goldens (flutter/flutter#141902) 2024-01-23 [email protected] Enable contextMenuBuilder in the absence of selectionControls (flutter/flutter#141810) 2024-01-23 [email protected] Roll Flutter Engine from b069d7f8f1fd to d3dbd4225e08 (3 revisions) (flutter/flutter#142005) 2024-01-23 98614782+auto-submit[bot]@users.noreply.github.com Reverts "hello_world app: migrate to Gradle Kotlin DSL" (flutter/flutter#142018) 2024-01-22 [email protected] Floating cursor docs (flutter/flutter#133002) 2024-01-22 [email protected] refactor: Rename filterPluginsByPlatform, cleanup Platform Strings (flutter/flutter#141780) 2024-01-22 [email protected] hello_world app: migrate to Gradle Kotlin DSL (flutter/flutter#141541) 2024-01-22 [email protected] Roll Flutter Engine from b2762f410840 to b069d7f8f1fd (4 revisions) (flutter/flutter#141993) 2024-01-22 [email protected] Remove duplicate code as suggested by natebosch. (flutter/flutter#141988) 2024-01-22 [email protected] Revert "Remove hack from PageView." (flutter/flutter#141977) 2024-01-22 [email protected] Roll Flutter Engine from d653559ae183 to b2762f410840 (3 revisions) (flutter/flutter#141978) 2024-01-22 [email protected] Do not hang on test failures of tests within `flutter_tools` (flutter/flutter#141821) 2024-01-22 [email protected] Remove unneeded expectation in test (flutter/flutter#141822) 2024-01-22 [email protected] Roll Flutter Engine from 1efe8ba6bc04 to d653559ae183 (3 revisions) (flutter/flutter#141972) 2024-01-22 [email protected] Add documentation which explains that `debugPrint` also logs in release mode (flutter/flutter#141595) 2024-01-22 [email protected] Fix `RangeSlider` throws a null-check error after `clearSemantics` is called (flutter/flutter#141965) 2024-01-22 [email protected] Roll Flutter Engine from c49989292fc1 to 1efe8ba6bc04 (1 revision) (flutter/flutter#141969) 2024-01-22 [email protected] [web] - Fix broken `TextField` in semantics mode when it's a sibling of `Navigator` (flutter/flutter#138446) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC [email protected],[email protected],[email protected] on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
Fixes #141823
Before this change, when a test would fail, the terminal would hang (by default for 30s) until killed by the test runner.
Basically,
runZonedGuardeddoes document (though not clearly) that a returned future should not be awaited:For example, you can see other places in Dart and Flutter that we circumvent that problem:
flutter/packages/flutter_tools/test/general.shard/base/async_guard_test.dart
Lines 279 to 306 in 5987563
I'm open to suggestions on how to test this :)
/cc @natebosch @jakemac53 @lrhn if you have any color commentary for us.