Repository navigation
Diagnose browser connection failures and time them out with the suite load timeout - #2788
Merged
Merged
Conversation
A wasm integration test failed with "Timed out waiting for Firefox to connect." and an empty "Browser output:". All three connection attempts had failed, but the error only showed the last attempt's output, and Firefox writes almost nothing to stdout or stderr, so there was no way to tell how far the browser got. When the connection times out, report every attempt instead of only the last: how long it waited, whether the browser process was still running or its exit code, the command used to start it, its output, and the requests it made to the test server. A new `BrowserRequestLog` wraps each compiler's server and records the host page, the resources it loads, and WebSocket upgrades, matched to the browser manager by the `managerUrl` query parameter or the `Referer` header. The report labels requests by file name so the server secret is not printed. Set Firefox preferences that print page console output and `dump()` calls to stdout, so errors in the host page show up in the browser output.
PR HealthChangelog Entry ✔️
Changes to files need to be accounted for in their respective changelogs. This check can be disabled by tagging the PR with |
# Conflicts: # pkgs/test/CHANGELOG.md # pkgs/test/pubspec.yaml
A browser had 30 seconds to connect, and one that didn't was replaced, up to 3 attempts, regardless of `suite_load_timeout`. #2786 gave browser suite loads until shortly before the suite load timeout instead, so the connection is the last part of loading a browser suite with its own fixed limit. Give the connection the same budget: until shortly before the suite load timeout, or no limit without one. A browser that times out uses the whole budget, so it isn't replaced. Timeouts were the only failure that was retried, so instead retry a browser that starts and then exits before connecting, up to 3 attempts while time remains. A process that fails to start, such as one with a missing executable, would fail the same way again and isn't retried. Each failed attempt is described in the error, including the output of browsers that exited. Without a suite load timeout a browser that never connects now hangs the run instead of failing, so add a `suite_load_timeout` to the wasm and spawn_hybrid integration tests, which run in browsers on CI.
natebosch
changed the title
The report for a browser that exited before connecting waited on `Browser.output`, but `Browser` closes that stream as soon as the process exits with an error, before its stdout and stderr have necessarily been read. On CI the reports for exited browsers said "Browser output: none" and the new tests that expect the output failed. Add `Browser.outputDrained`, which completes once both streams have closed and all of their lines are in `accumulatedOutput`, or the browser is closed, and wait on that instead.
The command line used to start a browser, which connection failure reports include, was recorded in an `Expando` keyed by the process, because each browser's launch function runs inside its `super(...)` call, before the subclass can store anything. Have the `startBrowser` callback return the command line alongside the process instead, and store it on `Browser`. This removes the `Expando` and the `startBrowserProcess` helper, and Safari now reports the `safaridriver` command line too.
Connecting to the browser and loading the suite in it each time out shortly before the suite load timeout. They are consecutive steps of loading a suite rather than nested, so they share one deadline, and loading gets whatever time connecting left. Say so where the platform starts both steps and on `browserLoadTimeout`, and note that only the suite that starts the shared browser times out connecting.
The connect timeout went from a Duration to a DateTime deadline and back to a Duration for each attempt, and the request log stamped requests with `DateTime.now()`. Wall clock time can jump, and the suite load timeout these are meant to beat is measured on a monotonic clock. Pass the remaining time to each attempt as a Duration, measuring each attempt with a Stopwatch, and measure request times in the log on its own Stopwatch. Attempt reports take the requests to describe instead of the log and start time.
Use `Uri` and `Uri.http` instead of parsing hardcoded strings in the browser manager and request log tests, and pass the `managerUrl` query parameter to the constructor instead of calling `replace`.
brianquinlan
approved these changes
Oct 15, 2026
copybara-service Bot
pushed a commit
to dart-lang/sdk
that referenced
this pull request
Oct 28, 2026
…est, tools, webkit_inspection_protocol Revisions updated by `dart tools/rev_sdk_deps.dart`. core (https://github.com/dart-lang/core/compare/04da676..d7bd45c): d7bd45c1 Wed Oct 28 13:23:54 2026 +0530 Chandra ObulaReddy [args] Wrap separators to usageLineLength (dart-lang/core#1032) 6903c6c7 Fri Oct 23 20:10:24 2026 +0000 Jonas Finnemann Jensen [crypto] Prepare to publish 3.0.8 (dart-lang/core#1026) 7c1c3700 Fri Oct 23 11:14:15 2026 -0700 Kevin Moore Tighten pull_request_label workflow permissions (dart-lang/core#1029) f01024fc Thu Oct 22 16:31:50 2026 -0700 Kevin Moore [collection] Share Iterable and Set tests across wrapper_test.dart (dart-lang/core#1028) c782eab4 Thu Oct 22 13:25:34 2026 +0200 Jonas Finnemann Jensen Update convert package README and add examples (dart-lang/core#999) dartdoc (https://github.com/dart-lang/dartdoc/compare/c0e19cf..db94437): db94437c Wed Oct 28 11:37:17 2026 +0100 Sarah Zakarias Add the doc-reference-target-differs warning (dart-lang/dartdoc#4306) b6e3980c Tue Oct 27 14:59:21 2026 +0100 Sarah Zakarias Only treat a doc comment as library documentation when it precedes a library directive (dart-lang/dartdoc#4303) 84a0141f Tue Oct 27 11:22:04 2026 +0100 Sarah Zakarias Add the unscoped-doc-reference warning (dart-lang/dartdoc#4305) b25bcd99 Thu Oct 22 15:56:17 2026 +0300 Sergey G. Grekhov Update/replace TODO in co19 tests (dart-lang/dartdoc#4299) 1e9e771e Thu Oct 22 12:18:48 2026 +0300 Sergey G. Grekhov `#4134`. Add more placement tests for `library` directives (dart-lang/dartdoc#4297) 17629ced Wed Oct 21 10:29:05 2026 -0700 Bob Nystrom Use newer function type syntax for parameters. (dart-lang/dartdoc#4290) f5928b1d Wed Oct 21 15:44:39 2026 +0300 Sergey G. Grekhov `#4134`. Add placement tests for enum values (dart-lang/dartdoc#4223) 24ae37f3 Wed Oct 21 11:18:07 2026 +0200 dependabot[bot] Bump test_reflective_loader from 0.4.0 to 0.6.0 (dart-lang/dartdoc#4296) ecosystem (https://github.com/dart-lang/ecosystem/compare/0ab0cfd..24b9ba8): 24b9ba8 Mon Oct 26 18:40:19 2026 -0700 Kevin Moore fix(repo_manage): increment patch version when transitioning to -wip (dart-lang/ecosystem#474) e1d4a04 Fri Oct 23 20:18:20 2026 +0000 dependabot[bot] Bump the github-actions group across 1 directory with 8 updates (dart-lang/ecosystem#473) 4b44357 Fri Oct 23 11:13:26 2026 -0700 Kevin Moore Tighten pull_request_label workflow permissions (dart-lang/ecosystem#472) i18n (https://github.com/dart-lang/i18n/compare/1f5ea2f..4c7b4ad): 4c7b4ad1 Fri Oct 23 11:14:00 2026 -0700 Kevin Moore Tighten pull_request_label workflow permissions (dart-lang/i18n#1113) 899251b1 Thu Oct 8 11:30:18 2026 +0200 Moritz [intl4x] Fix doc comment examples and verify them via {@example} in tests (dart-lang/i18n#1107) 5c8eb9d5 Thu Oct 8 10:27:58 2026 +0200 Moritz [intl4x] Fix YearStyle.withEra on ECMA when length is short or omitted (dart-lang/i18n#1105) b80bb04a Wed Sep 23 14:37:42 2026 +0200 Michael Thomsen Remove experiment flag no longer needed (dart-lang/i18n#1097) 766fd507 Wed Sep 23 13:59:56 2026 +0200 Moritz [intl4x] Mark DisplayNames API as @experimental and trigger CI on PR base edits (dart-lang/i18n#1098) 08de2f6f Fri Sep 18 12:49:50 2026 +0200 Moritz Rev package:intl4x to 1.0.0 and update status to preview (dart-lang/i18n#1088) native (https://github.com/dart-lang/native/compare/3cbe319..e0ca7c6): e0ca7c60f Wed Oct 28 12:08:25 2026 +0100 Michael Goderbauer [infra] Pin ubuntu-24.04 in workflows installing clang-15 (dart-lang/native#3771) eb09430a9 Tue Oct 27 10:43:22 2026 +0100 Michael Goderbauer [jni] Publish v1.1.1 (dart-lang/native#3765) 94b5f4337 Tue Oct 27 10:21:16 2026 +0100 Michael Goderbauer [jnigen] Pin ubuntu-24.04 in CI workflow (dart-lang/native#3766) 74cc078ee Tue Oct 27 09:29:23 2026 +0100 Michael Goderbauer [jni] Add ProGuard consumer rules for Kotlin classes used by suspend functions (dart-lang/native#3756) 0aa672fed Fri Oct 23 11:13:43 2026 -0700 Kevin Moore [infra] Tighten pull_request_label workflow permissions (dart-lang/native#3764) 8b11b6e01 Fri Oct 23 13:38:47 2026 +0200 Michael Goderbauer [jnigen] Bump AGP to 9.1.1 and KGP to 2.4.10 for Flutter beta (dart-lang/native#3758) protobuf (https://github.com/dart-lang/protobuf/compare/a73edc9..1d52301): 1d52301 Wed Oct 28 08:18:02 2026 -0700 Kevin Moore test(protoc_plugin): replace dart:mirrors in reserved_names_test (google/protobuf.dart#1104) shelf (https://github.com/dart-lang/shelf/compare/b66665b..73b2382): 73b2382 Thu Oct 22 10:43:29 2026 -0700 Kevin Moore shelf_router_generator: ignore invalid_use_of_internal_member on RouterEntry (dart-lang/shelf#548) shelf is held back at 73b2382, before dart-lang/shelf#547, because `pkgs/shelf/test/shelf_io_test.dart` now imports the unpublished `package:_shelf_compliance`, which breaks the shared `shelf_tests` target in google3. test (https://github.com/dart-lang/test/compare/d1afeb1..5967927): 5967927e Mon Oct 26 17:50:52 2026 -0700 Nate Bosch Diagnose browser connection failures and time them out with the suite load timeout (dart-lang/test#2788) 0692e8ae Mon Oct 26 16:29:27 2026 -0700 Kevin Moore Remove mirror_matchers.dart from package:matcher (dart-lang/test#2802) da21f85a Fri Oct 23 11:13:38 2026 -0700 Kevin Moore Tighten pull_request_label workflow permissions (dart-lang/test#2804) e18fd112 Thu Oct 22 15:38:51 2026 -0700 Nate Bosch Report tests that pass on retry as flaky (dart-lang/test#2801) b4ef80c0 Fri Oct 23 02:44:25 2026 +0600 Pranta Dutta Fix compact reporter crash when the terminal is too narrow (dart-lang/test#2799) e3a7f2b6 Tue Oct 20 15:51:34 2026 -0700 Nate Bosch Check that web entrypoints don't import VM-only libraries (dart-lang/test#2797) tools (https://github.com/dart-lang/tools/compare/eee87b1..3980dca): 3980dca39 Wed Oct 28 11:43:38 2026 +0100 Jonas Finnemann Jensen [html] Move contributors list from LICENSE to AUTHORS (dart-lang/tools#2733) 2866a7745 Wed Oct 28 11:13:19 2026 +0100 Jonas Finnemann Jensen [extension_discovery] Prepare for v2.1.1 release (dart-lang/tools#2727) 530feb74e Wed Oct 28 13:38:07 2026 +0530 Chandra ObulaReddy [csslib] Fix the aliceblue, antiquewhite and peru color values (dart-lang/tools#2732) d5c210271 Tue Oct 27 15:07:32 2026 -0700 Kevin Moore [source_span] Fix CR/CRLF highlighting, point-span context, and subspan (dart-lang/tools#2634) cbea5b08e Tue Oct 27 15:07:27 2026 -0700 Kevin Moore [string_scanner] Validate error() span bounds and test eager lastSpan (dart-lang/tools#2715) 22fd7a6a8 Tue Oct 27 15:07:20 2026 -0700 Kevin Moore [csslib] Fix parser, analyzer, polyfill, and printer crashes (dart-lang/tools#2638) webkit_inspection_protocol (https://github.com/google/webkit_inspection_protocol.dart/compare/762115a..2c255f2): 2c255f2 Thu Oct 22 14:18:44 2026 -0700 Kevin Moore Remove dart:mirrors and deprecate dom_model.dart and forwarder.dart (google/webkit_inspection_protocol.dart#141) web is held back at 4e60e61, before dart-lang/web#596, because it changes `package:web` signatures (e.g. `innerHTML=`, `parseFromString`, `Worker()`) from `JSAny` to `String` in ways that break callers in google3. Change-Id: Ia4253b35a91d7c05e34fe0e150ea4cb3e6628015 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/569660 Commit-Queue: Nate Bosch <[email protected]> Reviewed-by: Brian Quinlan <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A wasm integration test failed with "Timed out waiting for Firefox to
connect." and an empty "Browser output:"
(https://github.com/dart-lang/test/actions/runs/37977193562/job/113978908374).
All three connection attempts had failed, but the error only showed the last
attempt's output, and Firefox writes almost nothing to stdout or stderr, so
there was no way to tell how far the browser got.
Diagnostics
When a browser fails to connect, describe each attempt: how long it waited,
whether the browser process was still running or its exit code, the command
used to start it, its output, and the requests it made to the test server. A
new
BrowserRequestLogwraps each compiler's server and records the hostpage, the resources it loads, and WebSocket upgrades, matched to the browser
manager by the
managerUrlquery parameter or theRefererheader. Requestsare labeled by file name so the server secret is not printed.
Set Firefox preferences that print page console output and
dump()calls tostdout, so errors in the host page show up in the browser output.
Connect timeout
A browser had 30 seconds to connect, and one that didn't was replaced, up to
3 attempts, regardless of
suite_load_timeout. Following #2786, give theconnection until shortly before the suite load timeout instead, or no limit
without one. Connecting and then loading the suite are consecutive steps of
loading a suite, so they share that deadline, and loading gets whatever time
connecting left. A browser that times out uses the whole budget, so it isn't
replaced. Timeouts were the only failure that was retried; instead, retry a
browser that starts and then exits before connecting, up to 3 attempts while
time remains. A process that fails to start, such as one with a missing
executable, isn't retried.
Without a suite load timeout a browser that never connects now hangs the run
instead of failing, so add
suite_load_timeout: 2mto the wasm andspawn_hybrid integration tests, which run in browsers on CI.