Skip to content

Tag browser and node tests, relax flaky timeouts - #2770

Merged
natebosch merged 4 commits into
masterfrom
test-timeouts-and-tags
Sep 28, 2026
Merged

natebosch merged 4 commits into
masterfrom
test-timeouts-and-tags

Conversation

@natebosch

Copy link
Copy Markdown
Member

Several tests launch Chrome, Firefox, or Node without the matching tag,
so they miss the longer timeout, and for browsers the CI retries,
configured in dart_test.yaml. These account for most of the recent
browser flakes in CI. Find them by running with -x browser -x dart2js -x node and nonexistent browser executables, and tag everything that
still launches a browser or node.

  • Tag browser and node groups in compiler_runtime_matrix_test.dart.
  • Tag the Chrome tests in compiler_test.dart,
    configuration/compiler_test.dart, coverage_test.dart,
    github_reporter_test.dart, and line_and_col_test.dart, and the
    node test in line_and_col_test.dart.
  • Allow node tagged tests to run on macOS as well as linux.
  • Tag the coverage tests which run pub get with pub, and retry pub
    tagged tests on CI since resolution sometimes stalls on the network.
  • Raise the cross_compiler_hang suite load timeout from 20s to 60s.
    Chrome suites take 9-18s to load on the Windows runners while the exe
    suites compile concurrently. A regression of the deadlock the package
    reproduces hangs indefinitely, so a longer timeout still catches it.

Several tests launch Chrome, Firefox, or Node without the matching tag,
so they miss the longer timeout, and for browsers the CI retries,
configured in `dart_test.yaml`. These account for most of the recent
browser flakes in CI. Find them by running with `-x browser -x dart2js
-x node` and nonexistent browser executables, and tag everything that
still launches a browser or node.

- Tag browser and node groups in `compiler_runtime_matrix_test.dart`.
- Tag the Chrome tests in `compiler_test.dart`,
  `configuration/compiler_test.dart`, `coverage_test.dart`,
  `github_reporter_test.dart`, and `line_and_col_test.dart`, and the
  node test in `line_and_col_test.dart`.
- Allow `node` tagged tests to run on macOS as well as linux.
- Tag the coverage tests which run `pub get` with `pub`, and retry `pub`
  tagged tests on CI since resolution sometimes stalls on the network.
- Raise the `cross_compiler_hang` suite load timeout from 20s to 60s.
  Chrome suites take 9-18s to load on the Windows runners while the exe
  suites compile concurrently. A regression of the deadlock the package
  reproduces hangs indefinitely, so a longer timeout still catches it.
@natebosch
natebosch requested a review from a team as a code owner September 25, 2026 23:07
@github-actions github-actions Bot added type-infra A repository infrastructure change or enhancement package:test labels Sep 25, 2026
@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

PR Health

Changelog Entry ✔️
Package Changed Files

Changes to files need to be accounted for in their respective changelogs.

This check can be disabled by tagging the PR with skip-changelog-check.

@natebosch
natebosch requested a review from jakemac53 September 28, 2026 21:59
@natebosch
natebosch merged commit 52b4506 into master Sep 28, 2026
71 checks passed
@natebosch
natebosch deleted the test-timeouts-and-tags branch September 28, 2026 22:04
copybara-service Bot pushed a commit to dart-lang/sdk that referenced this pull request Oct 7, 2026
Revisions updated by `dart tools/rev_sdk_deps.dart`.

core (https://github.com/dart-lang/core/compare/08e4022..4d80610):
  4d80610a  Thu Oct 1 22:03:38 2026 +0000  dependabot[bot]  Bump the github-actions group with 5 updates (dart-lang/core#1005)

dartdoc (https://github.com/dart-lang/dartdoc/compare/2a040a8..5eb0f2c):
  5eb0f2c7  Thu Oct 1 19:03:48 2026 +0000  dependabot[bot]  Bump github/codeql-action/upload-sarif from 4.37.9 to 4.38.2 in the github-actions group (dart-lang/dartdoc#4291)

http (https://github.com/dart-lang/http/compare/d8e8e9c..585d433):
  585d433  Mon Oct 5 16:12:54 2026 -0700  Brian Quinlan  fix(cupertino_http): fix a websocket memory leak (dart-lang/http#1991)
  337adf3  Mon Oct 5 16:03:25 2026 -0700  Brian Quinlan  fix(cupertino_http): add `autoReleasePool` use to `CupertinoClient` (dart-lang/http#1990)
  47c57df  Tue Sep 29 14:07:01 2026 -0700  Brian Quinlan  fix(ok_http): fix `NetworkOnMainThreadException` (dart-lang/http#1995)
  d8cde5d  Tue Sep 29 14:06:44 2026 -0700  Brian Quinlan  fix test (dart-lang/http#1996)
  31056fe  Tue Sep 29 13:57:20 2026 -0700  Brian Quinlan  chore(cronet_http): prepare to release 1.10.0 (dart-lang/http#1993)
  9bc3a91  Tue Sep 29 13:53:25 2026 -0700  Brian Quinlan  fix(cronet_http): one global reference per `onReadCompleted` (dart-lang/http#1992)
  49ecd36  Mon Sep 28 14:20:26 2026 -0700  Brian Quinlan  fix(conformance): `supportsAbort` not considered in response (dart-lang/http#1994)

native (https://github.com/dart-lang/native/compare/27f7678..a2a6179):
  a2a617910  Tue Oct 6 13:53:17 2026 +0200  Daco Harkes  [jni_flutter] Bump version for publishing (dart-lang/native#3730)
  d3a99d66b  Tue Oct 6 19:46:34 2026 +1100  Liam Appelbe  [infra] Fix ffigen and jni_flutter (dart-lang/native#3724)
  b66d0815a  Tue Oct 6 04:28:35 2026 +0300  Manar Elhabbal  [ffigen] add Support passing C++ classes by value  (dart-lang/native#3677)
  f7ffe9667  Fri Oct 2 11:33:49 2026 +0200  Moritz  [native_toolchain_c] Don't pass /INCLUDE per symbol with a generated .def on Windows (dart-lang/native#3718)
  77a5e4169  Thu Oct 1 10:56:45 2026 +0300  Hassnaa Mohamed  [ffigen] Fix cpp method filtering (dart-lang/native#3696)
  cca065ca6  Thu Oct 1 10:17:12 2026 +1000  Liam Appelbe  [infra] Prepare to publish everything (dart-lang/native#3716)
  f13623ee3  Thu Oct 1 10:16:39 2026 +1000  Liam Appelbe  [jnigen] API docs (dart-lang/native#3715)
  be618790c  Thu Oct 1 10:05:26 2026 +1000  Liam Appelbe  [objective_c] API docs (dart-lang/native#3708)
  a18dd1148  Thu Oct 1 10:04:38 2026 +1000  Liam Appelbe  [ffigen] API docs (dart-lang/native#3707)
  6fd87c3b2  Tue Sep 29 19:01:11 2026 -0700  Brian Quinlan  [jni] Clarify `JBuffer.asUint8List` behavior (dart-lang/native#3704)
  6d5bbcfad  Wed Sep 30 09:35:47 2026 +1000  Liam Appelbe  [jni] API documentation (dart-lang/native#3706)
  fbcbc894b  Wed Sep 30 09:35:34 2026 +1000  Liam Appelbe  [ffigen] Document autorelease issues (dart-lang/native#3705)
  f99449ec9  Tue Sep 29 09:39:27 2026 +1000  Liam Appelbe  [infra] Add swift2objc to the workspace (dart-lang/native#3700)
  31494fa08  Tue Sep 29 09:38:59 2026 +1000  Liam Appelbe  [infra] Add jni_util to the workspace (dart-lang/native#3698)

shelf (https://github.com/dart-lang/shelf/compare/e5c8dc6..b2cb503):
  b2cb503  Wed Sep 30 22:53:07 2026 -0700  Kevin Moore  Resolve shelf.io.connection_info lazily (dart-lang/shelf#542)
  b70fafa  Tue Sep 29 13:37:08 2026 -0700  Kevin Moore  Match parameterless shelf_router routes without running a regex (dart-lang/shelf#544)

test (https://github.com/dart-lang/test/compare/4f92d53..d749b10):
  d749b104  Mon Oct 5 15:42:46 2026 -0700  Nate Bosch  Fail isolate suites when the test isolate exits (dart-lang/test#2775)
  da2157e9  Mon Oct 5 11:12:43 2026 -0700  Nate Bosch  Wait longer for the Chrome test tab to appear (dart-lang/test#2771)
  00bd0c99  Mon Oct 5 18:50:03 2026 +0300  Yusuf İhsan Görgel  Fix broken links in architecture.md (dart-lang/test#2774)
  3be2f547  Thu Oct 1 03:46:29 2026 +0000  dependabot[bot]  Bump the github-actions group with 4 updates (dart-lang/test#2773)
  40a5ad90  Tue Sep 29 12:07:31 2026 -0700  Nate Bosch  Delete compilation artifacts once they are no longer needed (dart-lang/test#2764)
  171ad0e5  Mon Sep 28 17:29:48 2026 -0700  Jacob MacDonald  export additional types from package:test_api/backend.dart (dart-lang/test#2745)
  52b45064  Mon Sep 28 15:04:08 2026 -0700  Nate Bosch  Tag browser and node tests, relax flaky timeouts (dart-lang/test#2770)

tools (https://github.com/dart-lang/tools/compare/d87eaf7..a66d517):
  a66d517d  Tue Oct 6 09:41:32 2026 -0700  Phil Quitslund  [analytics] `DASH__SUPPRESS_ANALYTICS` env handling (dart-lang/tools#2641)
  00c6e4ad  Fri Oct 2 17:17:49 2026 +0530  shivytyahoo  Fix VersionConstraint.intersection throwing on disjoint ranges (dart-lang/tools#2622)
  d523e1f8  Thu Oct 1 11:09:16 2026 +0200  Morgan :)  [unified_analytics] Append to the log file instead of rewriting it (dart-lang/tools#2613)
  6993cbad  Wed Sep 30 14:09:32 2026 +0300  Yusuf İhsan Görgel  [yaml] Improve error messages when mixing list and key-value syntax (dart-lang/tools#2610)
  068119b4  Tue Sep 29 23:21:03 2026 +0200  Lukas Klingsbo  [pubspec_parse] Support tag_pattern for git dependencies (dart-lang/tools#2565)
  7d859ed5  Mon Sep 28 20:28:23 2026 -0700  Kevin Moore  [api_summary] Add expectApiSummaryClean and --write/--check CLI flags (dart-lang/tools#2603)

web (https://github.com/dart-lang/web/compare/ec71a4c..63e6662):
  63e6662  Sat Oct 3 00:23:21 2026 +0000  Natalie Weizenbaum  Add a JSPair type (dart-lang/web#586)
  2d68d90  Fri Oct 2 20:26:48 2026 +0000  Natalie Weizenbaum  Use `js_interop` APIs in `js_interop_gen` (dart-lang/web#585)
  037823c  Thu Oct 1 21:59:21 2026 +0000  Natalie Weizenbaum  Add JSMap, JSSet, and JSSetLike types (dart-lang/web#571)
  b2d1d6b  Thu Oct 1 06:57:03 2026 +0000  dependabot[bot]  Bump the github-actions group with 3 updates (dart-lang/web#584)
  8dc8f75  Wed Sep 30 08:12:18 2026 -0700  Kevin Moore  [js_interop_gen] Split union/intersection AST types and extract TypeTransformer (dart-lang/web#580)
  751ccdf  Wed Sep 30 08:12:18 2026 -0700  Kevin Moore  [js_interop_gen] Decompose `Transformer._transformType` and deduplicate type caching (dart-lang/web#579)
  6f3dd4e  Wed Sep 30 08:12:17 2026 -0700  Kevin Moore  [js_interop_gen] Deduplicate `UnionType` and `IntersectionType` via `UnionOrIntersectionType` (dart-lang/web#578)
  d9f095e  Tue Sep 29 12:17:29 2026 -0700  Kevin Moore  [js_interop_gen] Remove unused JS interop extension types and bindings (dart-lang/web#577)
  07111ee  Tue Sep 29 09:51:36 2026 -0700  Kevin Moore  fix(js_interop_gen): update js_type_supertypes for Dart 3.12 (dart-lang/web#582)

Change-Id: I4532049c45561fe8f8c57713ac8ec2153096fd9a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/560660
Commit-Queue: Nate Bosch <[email protected]>
Reviewed-by: Brian Quinlan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

package:test type-infra A repository infrastructure change or enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants