You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 88202ad
Browse filesBrowse the repository at this point in the historyBrowse files
Running all setupAll hooks is expensive when what we are trying to do is to minimize the time spent for each shard.
This allows choosing a less fine-grained strategy of sharding by individual test files.
Fixesdart-lang#2614
dart test --total-shards 3 --shard-index 1 path/to/test.dart
205
205
dart test --total-shards 3 --shard-index 2 path/to/test.dart
206
206
```
207
-
Sharding: This refers to the process of splitting up a large test suite into
208
-
smaller subsets (called shards) that can be run independently. Sharding is
209
-
particularly useful for distributed testing, where multiple machines are used
210
-
to run tests simultaneously. By dividing the test suite into smaller subsets,
211
-
you can run tests in parallel across multiple machines, which can significantly
212
-
reduce the overall testing time.
207
+
208
+
By default, sharding is done by individual tests within each suite.
209
+
You can shard by entire test suites (files) using the `--shard-by-suite` flag.
210
+
211
+
* When sharding by test (default): Distribute individual test cases across
212
+
shards. This balances the test load at the test case level. Test cases from
213
+
each suite are sliced continuously, which minimizes how often a suite is
214
+
split across shards and helps maximize the re-use of suite `setUpAll` and
215
+
`tearDownAll` setups.
216
+
* When sharding by suite (using `--shard-by-suite`): Distribute entire test suites
217
+
across shards. This can be faster for projects with many small test suites as it
218
+
avoids loading every suite in every shard. Because test suites are not split,
219
+
any `setUpAll` and `tearDownAll` setups within a suite (including shared helper setups imported across suites) run only on the shard executing that suite.
220
+
221
+
Sharding is particularly useful for distributed testing, where multiple
222
+
machines are used to run tests simultaneously. By dividing the test suite into
223
+
smaller subsets, you can run tests in parallel across multiple machines, which
224
+
can significantly reduce the overall testing time.
225
+
226
+
### Interaction with Test Filters
227
+
228
+
The sharding modes interact differently with filters like `--name` or `--tags`:
229
+
230
+
* When sharding by test case (default), the partition is calculated *after* applying filters. This guarantees matching test cases are distributed as evenly as possible across all shards.
231
+
* When sharding by test suite (using `--shard-by-suite`), suite-level annotations (such as `@TestOn` platform selectors and suite-level `@Tags` at the top of a file) are evaluated before sharding to filter out non-matching suites. However, test filters which apply to individual test cases or groups (such as `--name`) will be reflected in the distribution when sharding by test case, but not when sharding by test suite. Test invocations with an uneven number of matching test cases between test suites can cause uneven workloads across the shards.
0 commit comments