Skip to content

Never retry tearDownAll - #2815

Merged
natebosch merged 1 commit into
masterfrom
teardownall-no-retry
Nov 10, 2026
Merged

natebosch merged 1 commit into
masterfrom
teardownall-no-retry

Conversation

@natebosch

Copy link
Copy Markdown
Member

A retried tearDownAll didn't run any of its callbacks again.
Invoker.runTearDowns removes each callback from the declarer's list
before running it, so by the time a retry started the list was empty. The
retry always passed, which hid the original failure.

Give the synthetic (tearDownAll) test retry: 0 so a failure fails the
run, whatever retry the group, suite, or tags configure.
Platform-specific metadata, such as on_platform in dart_test.yaml, is
merged into the test's metadata later by forPlatform, so clear the retry
in those entries too. setUpAll still retries.

We chose not to make tearDownAll retries work. Re-running cleanup
correctly means tracking which callbacks failed, including ones that time
out or fail asynchronously, and the complexity cost is higher than the
utility. Teardown callbacks are often not safe to run twice anyway. A
tearDownAll callback that can fail on flaky cleanup should handle that
failure itself, for example by retrying within the callback.

This only surfaces failures that a retry was already hiding.

Fixes #2800

A retried `tearDownAll` didn't run any of its callbacks again.
`Invoker.runTearDowns` removes each callback from the declarer's list
before running it, so by the time a retry started the list was empty. The
retry always passed, which hid the original failure.

Give the synthetic `(tearDownAll)` test `retry: 0` so a failure fails the
run, whatever `retry` the group, suite, or tags configure.
Platform-specific metadata, such as `on_platform` in `dart_test.yaml`, is
merged into the test's metadata later by `forPlatform`, so clear the retry
in those entries too. `setUpAll` still retries.

We chose not to make `tearDownAll` retries work. Re-running cleanup
correctly means tracking which callbacks failed, including ones that time
out or fail asynchronously, and the complexity cost is higher than the
utility. Teardown callbacks are often not safe to run twice anyway. A
`tearDownAll` callback that can fail on flaky cleanup should handle that
failure itself, for example by retrying within the callback.

This only surfaces failures that a retry was already hiding.

Fixes #2800
@natebosch
natebosch requested a review from a team as a code owner November 6, 2026 18:07
@natebosch
natebosch requested a review from jakemac53 November 6, 2026 18:07
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown

PR Health

Publish ✔️
Package Version Status Publish tag (post-merge)
package:test ⭐ 1.34.0-wip WIP (no publish necessary)
package:test_api ⭐ 0.7.16-wip WIP (no publish necessary)
  • 3 already published.
  • 4 WIP (no publish necessary).

Documentation at https://github.com/dart-lang/ecosystem/wiki/Publishing-automation.

This check can be disabled by tagging the PR with skip-publish-check.

Changelog Entry ✔️
Package Changed Files

Changes to files need to be accounted for in their respective changelogs.

This check can be disabled by tagging the PR with skip-changelog-check (for example, for non-user-visible changes when the package version is already at -wip).

@natebosch
natebosch merged commit 3533ae1 into master Nov 10, 2026
70 checks passed
@natebosch
natebosch deleted the teardownall-no-retry branch November 10, 2026 23:06
copybara-service Bot pushed a commit to dart-lang/sdk that referenced this pull request Nov 12, 2026
Revisions updated by `dart tools/rev_sdk_deps.dart`.

core (https://github.com/dart-lang/core/compare/36f57a9..fa0c6b7):
  fa0c6b77  Fri Nov 6 12:56:58 2026 +0100  Michael Thomsen  Release os_detect 2.0.4 (dart-lang/core#1038)

dartdoc (https://github.com/dart-lang/dartdoc/compare/03ec6df..7e2508c):
  7e2508c6  Wed Nov 11 14:52:22 2026 +0200  Sergey G. Grekhov  Add constructor doc comment tests for enums and extension types (dart-lang/dartdoc#4316)
  03dfb467  Mon Nov 9 11:00:32 2026 +0200  Sergey G. Grekhov  `#3330`. Split test for parameters to separate tests (dart-lang/dartdoc#4314)
  23302d53  Mon Nov 9 10:56:34 2026 +0200  Sergey G. Grekhov  Add tests for primary constructors (dart-lang/dartdoc#4313)

ecosystem (https://github.com/dart-lang/ecosystem/compare/54e1673..1d2b5ff):
  1d2b5ff  Tue Nov 10 09:04:24 2026 -0800  Kevin Moore  feat(repo_manage): add --replace-pattern to report changelog (dart-lang/ecosystem#482)

http (https://github.com/dart-lang/http/compare/4646fe6..2bac218):
  2bac218  Fri Nov 6 17:49:23 2026 -0800  dart-ecosystem-bot[bot]  Tidy: package:cronet_http
  cdf967c  Fri Nov 6 15:40:49 2026 -0800  Brian Quinlan  chore(groundkeeper): use stable for flutter (dart-lang/http#2066)
  6582177  Thu Nov 5 13:04:35 2026 -0800  Brian Quinlan  doc: use timeout as an abort example (dart-lang/http#2062)

native (https://github.com/dart-lang/native/compare/6287745..b21ef2d):
  b21ef2da3  Wed Nov 11 16:07:34 2026 +0530  dhairya05  [json_syntax_generator] Pass the parameter name to ArgumentError.value (dart-lang/native#3787)
  20a607e48  Wed Nov 11 07:28:03 2026 -0300  Daco Harkes  [code_assets] [hooks] [hooks_runner] Adopt `.g.dart` for generated files (dart-lang/native#3790)
  cc974ddec  Wed Nov 11 19:59:39 2026 +1100  Liam Appelbe  [objective_c] Fix `isValidObject` bug on iOS simulator (dart-lang/native#3788)
  f7edbdee3  Sun Nov 8 13:18:25 2026 +1100  Liam Appelbe  [jni] Fix possible receiver NPEs (dart-lang/native#3784)
  0ca7d6745  Fri Nov 6 10:20:21 2026 -0300  Daco Harkes  [code_assets] [hooks_runner] [native_toolchain_c] Prepare for release (dart-lang/native#3781)
  baad15ab4  Fri Nov 6 02:56:52 2026 +0100  Kravchenko Igor  [objective_c] Link Foundation in the native support library (dart-lang/native#3782)

shelf (https://github.com/dart-lang/shelf/compare/9946de5..b175e9f):
  b175e9f  Tue Nov 10 09:14:31 2026 +0530  Muhammad Uvesh  shelf_static: ignore invalid If-Modified-Since headers (dart-lang/shelf#557)
  02b5f94  Sat Nov 7 23:09:28 2026 +0530  dhairya05  Fix typo in shelf_router docs (dart-lang/shelf#556)

test (https://github.com/dart-lang/test/compare/2a9e017..3533ae1):
  3533ae1e  Tue Nov 10 15:06:08 2026 -0800  Nate Bosch  Never retry tearDownAll (dart-lang/test#2815)
  5680b0b1  Fri Nov 6 12:52:11 2026 +0100  Sigurd Meldgaard  Add globalSetup and addGlobalTearDown APIs for cross-suite setup and teardown (dart-lang/test#2714)
  b8dc888a  Thu Nov 5 18:11:37 2026 -0800  Nate Bosch  [checks_codegen] Skip private fields and fields referencing private types (dart-lang/test#2782)
  66e062c4  Thu Nov 5 16:05:36 2026 -0800  Nate Bosch  [checks_codegen] Generate full field types with imports (dart-lang/test#2781)

tools (https://github.com/dart-lang/tools/compare/727fd9c..589d844):
  589d844c4  Wed Nov 11 20:27:32 2026 +0300  Hasan Armoush  fix(markdown): combine adjacent Text nodes when the element is last in its list (dart-lang/tools#2598)

Change-Id: I13acaa7378c86f2c33d88f3c4b163acf2b48ff08
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/576820
Commit-Queue: Nate Bosch <[email protected]>
Reviewed-by: Nicholas Shahan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Retrying a failed tearDownAll doesn't re-run its callbacks, so the retry always passes

2 participants