Skip to content

Conversation

@Swiftaxe
Copy link
Contributor

This PR fixed the problem that has prevented daemon_test.dart being shuffled. Part of #85160.

The issues

Two issues were found related to three tests. No test failed for me using the reported seed 1000. But it failed on seed 123.

Test A: 'notifyingLogger outputs trace messages in verbose mode'
Test B: 'notifyingLogger ignores trace messages in non-verbose mode'
Test C: 'notifyingLogger buffers messages sent before a subscription'

First Issue - Order Dependency

All three tests use the bufferLogger variable that is initialized inside group 'daemon'. As these
three tests are placed outside that group, they were failing if run before the group 'daemon'.

Second Issue - Broken Test and Order Dependency

As far as I can see, the test B has never been working as intended.
The line 'logger.printTrace('test');' should not be logged in bufferLogger in non-verbose mode. The test expected it to be logged.

It was working though because test A was logging the same string to bufferLogger and leaving the log behind for later tests. But if test B was run before test A, it would fail.

The fix

First issue

Create a new test group called 'notifyingLogger' to initialize bufferLogger and then clear it, after each of these tests. The bufferLogger variable was isolated in each group to avoid future mistakes.

Second Issue

Test B was changed to expect the bufferLogger to be empty.

Note

I could not reproduce any test failings on randomize ordering seed 1000. I found these issues using seed 123.
Any reviewer should consider testing seed 1000 to make sure there is no race condition that I cannot reproduce.

Pre-launch Checklist

  • I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
  • I read the [Tree Hygiene] wiki page, which explains my responsibilities.
  • I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement].
  • I signed the [CLA].
  • I listed at least one issue that this PR fixes in the description above.
  • I updated/added relevant documentation (doc comments with ///).
  • I added new tests to check the change I am making, or this PR is [test-exempt].
  • All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on [Discord].

@flutter-dashboard flutter-dashboard bot added the tool Affects the "flutter" command-line tool. See also t: labels. label Feb 23, 2022
Copy link
Contributor

@jonahwilliams jonahwilliams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

…shard/hermetic/daemon_test.dart

Co-authored-by: Jonah Williams <[email protected]>
@fluttergithubbot fluttergithubbot merged commit 47b5ed3 into flutter:master Feb 23, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 23, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/plugins that referenced this pull request Feb 23, 2022
@Swiftaxe Swiftaxe deleted the 85160_daemon_test branch February 23, 2022 21:50
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 23, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 23, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 23, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tool Affects the "flutter" command-line tool. See also t: labels.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants