Repository navigation
Commit 4f89c8f
authored
Allow resetting test invariants in
Fixes flutter#110488.
### Background: The `testWidgets` Lifecycle
Understanding this change requires distinguishing between two layers of
`Future`s and two distinct verification steps in `testWidgets`:
#### 1. Two Layers of `Future`s
* **`testBody()` Future**: The closure containing the user's test logic.
* **`runTest()` Future**: The overall async harness returned by
`binding.runTest()`. It manages the test zone, unmounts the widget tree,
and resolves only after `testBody()` and test-specific verifications
finish.
#### 2. Two Distinct Verification Phases
* **Test-specific verifications (`invariantTester`)**: Passed as a
callback to `runTest` and executed synchronously inside `_runTestBody`
immediately after the `testBody()` Future completes and the widget tree
is unmounted. In `testWidgets`, this runs
`tester._endOfTestVerifications` to verify that tickers and semantics
handles were disposed.
* **Framework-level invariant checks**: Verifies global framework state,
including ensuring foundation/rendering/widgets debug flags were reset
and no non-periodic timers or animations remain.
#### 3. The Teardown Lifecycle & Root Cause of flutter#110488
When `testWidgets` runs:
1. `testWidgets` registers `test_package.addTearDown(binding.postTest)`
with `package:test`.
2. `testWidgets` calls and returns `binding.runTest(...)`.
3. Inside `testBody()`, users may register cleanups using `addTearDown`
(for example, `addTearDown(() => debugDefaultTargetPlatformOverride =
null)`).
4. `package:test` executes teardowns in **LIFO (reverse registration)
order** only *after* the Future returned by `runTest` completes.
Previously, framework-level invariant checks ran at the end of
`_runTestBody` *before* the `runTest` Future completed. As a result,
they ran before `package:test` executed any user-registered
`addTearDown` callbacks, causing tests that clean up debug flags via
`addTearDown` to fail invariant verification (flutter#110488).
The following diagram shows the code flow after this PR:
```
testWidgets(description, (tester) async { ... })
│
├── 1. Registration Phase
│ └── test_package.addTearDown(binding.postTest) [Teardown Stack: flutter#1]
│
├── 2. Test Execution Phase: binding.runTest(...)
│ │
│ │ [ Future 1: testBody() ]
│ ├──► Run testBody(tester)
│ │ └── User calls addTearDown(...) [Teardown Stack: flutter#2]
│ │
│ │ [ Post-testBody cleanup (inside runTest) ]
│ ├──► Unmount widget tree (runApp)
│ ├──► Verification 1: invariantTester() (tickers, semantics)
│ └──► Set _shouldVerifyInvariants = true
│
│ [ Future 2: runTest() completes ]
│
└── 3. Teardown Phase: package:test executes in LIFO order
│
├──► [Step 1] User Teardowns (Stack flutter#2)
│ └── e.g., debugDefaultTargetPlatformOverride = null
│
└──► [Step 2] binding.postTest() (Stack flutter#1)
├── Verification 2: Framework Invariants (debug vars, timers)
├── Reset Binding State (_currentFakeAsync, etc.)
└── reportTestException(...) (if Verification 2 failed)
```
### Changes & Design Rationale
#### 1. Defer Framework Invariant Checks to `postTest`
Framework invariant checks (`_verifyInvariants()`,
`_verifyAutoUpdateGoldensUnset`, etc.) are moved from `_runTestBody`
into `TestWidgetsFlutterBinding.postTest`.
Because `testWidgets` registers `binding.postTest` before running the
test body, `package:test`'s LIFO teardown ordering guarantees that all
user-registered `addTearDown` callbacks execute before
`binding.postTest`.
#### 2. Clarifying the `postTest` Error Contract
Previously, `postTest` only performed state cleanup and did not check
invariants. Moving invariant checks into `postTest` introduced the
question of how invariant failures should be propagated: allowing
`postTest` to throw, or absorbing errors and reporting them via
`reportTestException`.
We explicitly define the contract of `postTest` to **not throw
exceptions**, reporting any invariant failures via
`reportTestException`:
* **Rationale & Trade-offs**:
* *Alternative considered (throwing)*: If `super.postTest()` threw on an
invariant failure, `package:test` would still capture the failure.
However, subclasses (`AutomatedTestWidgetsFlutterBinding`,
`LiveTestWidgetsFlutterBinding`, or custom bindings) override `postTest`
to reset their own state (e.g. nulling `_currentFakeAsync` / `_clock`,
or setting `_inTest = false`). Allowing `super.postTest()` to throw
would require every subclass to defensively wrap `super.postTest()` in
`try ... finally`. If any subclass omitted this, an invariant failure
would leave the binding in a dirty state and trigger cascading `!inTest`
failures in subsequent tests.
* *Chosen approach (non-throwing)*: Absorbing errors in `postTest` and
forwarding them to `reportTestException` guarantees that all cleanup in
base and subclass implementations executes sequentially without
defensive `try ... finally` boilerplate, while still cleanly failing the
test.
**Why this does not break existing behavior:**
In the previous implementation, when invariant assertions failed inside
`_runTestBody`, they were caught by `_testZone.handleUncaughtError` and
forwarded to `reportTestException` (which delegates to
`package:test.registerException`). They never escaped `runTest` as
unhandled thrown exceptions. Catching invariant failures in `postTest`
and routing them to `reportTestException` preserves this exact error
reporting pipeline, diagnostic output, and failure semantics without
disrupting teardown execution.
#### 3. Preserving Existing Behavior on Failed Tests
(`_shouldVerifyInvariants`)
Historically, `_runTestBody` only verified invariants if the test body
did not fail:
```dart
if (_pendingExceptionDetails == null) {
// We only try to clean up and verify invariants if we didn't already
// fail. If we got an exception already, then we instead leave everything
// alone so that we don't cause more spurious errors.
...
_verifyInvariants();
}
```
When invariant checks are moved to `postTest`,
`_pendingExceptionDetails` is no longer available because it has already
been reported and cleared by `testCompletionHandler` when `runTest`
completed.
To preserve this exact existing behavior across the lifecycle boundary,
`_shouldVerifyInvariants` is introduced. It is set to `true` at the end
of `_runTestBody` only when `_pendingExceptionDetails == null`. If a
test fails in its body, `_shouldVerifyInvariants` remains `false`, and
`postTest` skips invariant checks—ensuring no behavior change and
preventing spurious errors (e.g. active animations on an unmounted tree)
from masking the real failure.
#### 4. Documentation Updates
Updated the doc comments on:
* `TestWidgetsFlutterBinding.postTest`: Documents the non-throwing
contract, the `@mustCallSuper` requirement, and error reporting via
`reportTestException`.
* `TestWidgetsFlutterBinding.runTest`: Clarifies the distinction between
`invariantTester` (which runs before `runTest` completes) and framework
invariant checks (which run in `postTest` after `runTest` and user
teardowns complete).
### Tests
- Added unit tests in
`packages/flutter_test/test/bindings_invariants_test.dart` verifying
that foundation variables (`debugDefaultTargetPlatformOverride`,
`debugDoublePrecision`, `debugBrightnessOverride`),
`autoUpdateGoldenFiles`, `ErrorWidget.builder`, and
`shouldPropagateDevicePointerEvents` can all be reset using
`addTearDown`.
- Added test cases verifying invariant failures in `postTest` and
ensuring invariant checks are skipped when the test body throws.
## Pre-launch Checklist
- [ ] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [ ] I read the [AI contribution guidelines] and understand my
responsibilities, or I am not using AI tools.
- [ ] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [ ] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [ ] I signed the [CLA].
- [ ] I listed at least one issue that this PR fixes in the description
above.
- [ ] I updated/added relevant documentation (doc comments with `///`).
- [ ] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [ ] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [ ] All existing and new tests are passing.
If you need help, consider asking for advice on the #hackers-new channel
on [Discord].
If this change needs to override an active code freeze, provide a
comment explaining why. The code freeze workflow can be overridden by
code reviewers. See pinned issues for any active code freezes with
guidance.
**Note**: The Flutter team is currently trialing the use of [Gemini Code
Assist for
GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code).
Comments from the `gemini-code-assist` bot should not be taken as
authoritative feedback from the Flutter team. If you find its comments
useful you can update your code accordingly, but if you are unsure or
disagree with the feedback, please feel free to wait for a Flutter team
member's review for guidance on which automated comments should be
addressed.
<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview
[AI contribution guidelines]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#ai-contribution-guidelines
[Tree Hygiene]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
[test-exempt]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes
[Discord]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md
[Data Driven Fixes]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.mdaddTearDown (flutter#192082)1 parent e71a5b5 commit 4f89c8f
3 files changed
Lines changed: 275 additions & 44 deletions
File tree
- packages/flutter_test
- lib/src
- test
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1688 | 1688 | | |
1689 | 1689 | | |
1690 | 1690 | | |
1691 | | - | |
1692 | | - | |
| 1691 | + | |
| 1692 | + | |
| 1693 | + | |
| 1694 | + | |
| 1695 | + | |
| 1696 | + | |
| 1697 | + | |
| 1698 | + | |
| 1699 | + | |
1693 | 1700 | | |
1694 | 1701 | | |
1695 | 1702 | | |
| |||
1944 | 1951 | | |
1945 | 1952 | | |
1946 | 1953 | | |
| 1954 | + | |
| 1955 | + | |
| 1956 | + | |
| 1957 | + | |
| 1958 | + | |
1947 | 1959 | | |
1948 | 1960 | | |
1949 | 1961 | | |
1950 | 1962 | | |
1951 | 1963 | | |
1952 | 1964 | | |
1953 | 1965 | | |
1954 | | - | |
1955 | | - | |
1956 | | - | |
1957 | | - | |
1958 | | - | |
1959 | 1966 | | |
1960 | 1967 | | |
1961 | 1968 | | |
| |||
1974 | 1981 | | |
1975 | 1982 | | |
1976 | 1983 | | |
1977 | | - | |
1978 | | - | |
1979 | | - | |
1980 | | - | |
1981 | | - | |
| 1984 | + | |
| 1985 | + | |
1982 | 1986 | | |
1983 | 1987 | | |
1984 | 1988 | | |
1985 | 1989 | | |
1986 | 1990 | | |
1987 | 1991 | | |
| 1992 | + | |
| 1993 | + | |
| 1994 | + | |
| 1995 | + | |
1988 | 1996 | | |
1989 | 1997 | | |
| 1998 | + | |
| 1999 | + | |
| 2000 | + | |
| 2001 | + | |
| 2002 | + | |
| 2003 | + | |
| 2004 | + | |
| 2005 | + | |
| 2006 | + | |
| 2007 | + | |
| 2008 | + | |
| 2009 | + | |
| 2010 | + | |
| 2011 | + | |
| 2012 | + | |
| 2013 | + | |
| 2014 | + | |
| 2015 | + | |
| 2016 | + | |
| 2017 | + | |
| 2018 | + | |
| 2019 | + | |
| 2020 | + | |
| 2021 | + | |
| 2022 | + | |
| 2023 | + | |
| 2024 | + | |
| 2025 | + | |
| 2026 | + | |
| 2027 | + | |
| 2028 | + | |
| 2029 | + | |
| 2030 | + | |
| 2031 | + | |
| 2032 | + | |
| 2033 | + | |
| 2034 | + | |
1990 | 2035 | | |
1991 | 2036 | | |
1992 | 2037 | | |
| |||
2042 | 2087 | | |
2043 | 2088 | | |
2044 | 2089 | | |
2045 | | - | |
2046 | | - | |
2047 | | - | |
2048 | | - | |
2049 | | - | |
2050 | | - | |
2051 | | - | |
| 2090 | + | |
2052 | 2091 | | |
2053 | 2092 | | |
2054 | 2093 | | |
| |||
2062 | 2101 | | |
2063 | 2102 | | |
2064 | 2103 | | |
2065 | | - | |
2066 | | - | |
2067 | | - | |
2068 | | - | |
2069 | | - | |
2070 | | - | |
2071 | | - | |
| 2104 | + | |
2072 | 2105 | | |
2073 | 2106 | | |
2074 | 2107 | | |
| |||
2077 | 2110 | | |
2078 | 2111 | | |
2079 | 2112 | | |
2080 | | - | |
2081 | | - | |
2082 | | - | |
2083 | | - | |
2084 | | - | |
2085 | | - | |
2086 | | - | |
| 2113 | + | |
2087 | 2114 | | |
2088 | 2115 | | |
2089 | 2116 | | |
| |||
2092 | 2119 | | |
2093 | 2120 | | |
2094 | 2121 | | |
2095 | | - | |
2096 | | - | |
2097 | | - | |
2098 | | - | |
2099 | | - | |
2100 | | - | |
2101 | | - | |
2102 | | - | |
| 2122 | + | |
| 2123 | + | |
2103 | 2124 | | |
2104 | 2125 | | |
2105 | 2126 | | |
| |||
2137 | 2158 | | |
2138 | 2159 | | |
2139 | 2160 | | |
| 2161 | + | |
| 2162 | + | |
| 2163 | + | |
| 2164 | + | |
| 2165 | + | |
| 2166 | + | |
2140 | 2167 | | |
2141 | 2168 | | |
| 2169 | + | |
| 2170 | + | |
| 2171 | + | |
| 2172 | + | |
| 2173 | + | |
| 2174 | + | |
| 2175 | + | |
| 2176 | + | |
| 2177 | + | |
| 2178 | + | |
| 2179 | + | |
| 2180 | + | |
| 2181 | + | |
| 2182 | + | |
| 2183 | + | |
| 2184 | + | |
| 2185 | + | |
| 2186 | + | |
| 2187 | + | |
| 2188 | + | |
2142 | 2189 | | |
2143 | 2190 | | |
2144 | 2191 | | |
| |||
2174 | 2221 | | |
2175 | 2222 | | |
2176 | 2223 | | |
| 2224 | + | |
| 2225 | + | |
| 2226 | + | |
| 2227 | + | |
2177 | 2228 | | |
2178 | 2229 | | |
2179 | 2230 | | |
| |||
2528 | 2579 | | |
2529 | 2580 | | |
2530 | 2581 | | |
| 2582 | + | |
| 2583 | + | |
| 2584 | + | |
| 2585 | + | |
| 2586 | + | |
| 2587 | + | |
2531 | 2588 | | |
2532 | 2589 | | |
2533 | 2590 | | |
| |||
2549 | 2606 | | |
2550 | 2607 | | |
2551 | 2608 | | |
2552 | | - | |
2553 | 2609 | | |
2554 | 2610 | | |
2555 | 2611 | | |
| |||
0 commit comments