Skip to content

GPU process is terminated as unresponsive while it is still starting up - #80415

Open
yury-s wants to merge 1 commit into
WebKit:mainfrom
yury-s:eng/GPU-process-is-terminated-as-unresponsive-while-it-is-still-starting-up
Open

yury-s wants to merge 1 commit into
WebKit:mainfrom
yury-s:eng/GPU-process-is-terminated-as-unresponsive-while-it-is-still-starting-up

Conversation

@yury-s

@yury-s yury-s commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

fe90600

GPU process is terminated as unresponsive while it is still starting up
https://bugs.webkit.org/show_bug.cgi?id=331017

Reviewed by NOBODY (OOPS!).

GPUProcessProxy began its responsiveness checks in didFinishLaunching(), so the
responsiveness timer armed by createGPUProcessConnection() started as soon as
the process had launched. A GPU process that needed more than three seconds to
start and to handle InitializeGPUProcess, for example on a loaded machine, was
terminated as unresponsive although it was not hung. On the Windows port
didFinishLaunching() is called right after CreateProcess() returns, so the
whole startup of the process counted against the timeout.

Begin the responsiveness checks when the GPU process replies to
InitializeGPUProcess instead, like NetworkProcessProxy does since 248341@main.
A check requested before that is delayed until the reply arrives.

The reply handler is also called when the reply is cancelled, for example when
the process exits before it has initialized. Do not begin the checks in that
case, neither in GPUProcessProxy nor in NetworkProcessProxy.

A GPU process that hangs before it replies to InitializeGPUProcess is still
terminated: a web process which waits for it times out and reports that to the
UI process. That timeout is 15 seconds. Add the
WebProcessGPUProcessTimeoutForTesting preference to override it, so that the
test of this case does not have to wait that long.

Tests: Tools/TestWebKitAPI/Tests/WebKit/WKWebView/GPUProcess.mm

* Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml:
* Source/WebKit/UIProcess/GPU/GPUProcessProxy.cpp:
(WebKit::GPUProcessProxy::GPUProcessProxy):
(WebKit::GPUProcessProxy::didFinishLaunching):
* Source/WebKit/UIProcess/Network/NetworkProcessProxy.cpp:
(WebKit::NetworkProcessProxy::sendCreationParametersToNewProcess):
* Source/WebKit/WebProcess/WebProcess.cpp:
(WebKit::WebProcess::gpuProcessTimeoutDuration const):
(WebKit::WebProcess::updateProcessPreferences):
* Source/WebKit/WebProcess/WebProcess.h:
* Tools/TestWebKitAPI/Tests/WebKit/WKWebView/GPUProcess.mm:
(-[GPUProcessLaunchObserver observeValueForKeyPath:ofObject:change:context:]):
(TestWebKitAPI::TEST(GPUProcess, NotTerminatedWhenSlowToInitialize)):
(TestWebKitAPI::TEST(GPUProcess, TerminatedWhenHungBeforeInitialization)):

fe90600

Misc iOS, visionOS, tvOS & watchOS macOS Linux Windows
✅ 🧪 style ✅ 🛠 ios ✅ 🛠 mac ✅ 🛠 wpe ✅ 🛠 win
✅ 🧪 bindings ✅ 🛠 ios-sim-cmake ✅ 🛠 mac-cmake-debug ✅ 🧪 wpe-wk2 ✅ 🧪 win-tests
✅ 🧪 webkitperl ✅ 🧪 ios-cmake-wk2 ✅ 🧪 api-mac ✅ 🧪 api-wpe
✅ 🧪 ios-cmake-wk2-wpt ✅ 🧪 api-mac-cmake-debug ✅ 🧪 jsc-wpe
✅ 🧪 jsc-x86-64 ✅ 🧪 api-ios-cmake ✅ 🧪 mac-wk2 ✅ 🛠 gtk3-gcc
✅ 🛠 🧪 jsc-debug-arm64 ✅ 🛠 ios-safer-cpp ✅ 🧪 mac-cmake-debug-wk2 ✅ 🛠 gtk
✅ 🛠 vision ✅ 🧪 gtk-wk2
✅ 🛠 vision-sim ✅ 🧪 mac-intel-wk2 ✅ 🧪 api-gtk
✅ 🧪 vision-wk2 ✅ 🛠 mac-safer-cpp ✅ 🛠 playstation
✅ 🛠 tv ✅ 🧪 mac-site-isolation
✅ 🛠 tv-sim ⏳ 🧪 mac-webdriver
✅ 🛠 watch
✅ 🛠 watch-sim

@yury-s
yury-s requested a review from cdumez as a code owner November 11, 2026 01:00
@yury-s yury-s self-assigned this Nov 11, 2026
sendWithAsyncReply(Messages::GPUProcess::InitializeGPUProcess(WTF::move(parameters)), [initializationActivityAndGrant = initializationActivityAndGrant()] { });
sendWithAsyncReply(Messages::GPUProcess::InitializeGPUProcess(WTF::move(parameters)), [weakThis = WeakPtr { *this }, initializationActivityAndGrant = initializationActivityAndGrant()] {
if (RefPtr protectedThis = weakThis)
protectedThis->beginResponsivenessChecks();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For style points you could add a check if the async reply is a cancel reply (here and in network process), so that it's consistent ("if the reply is answered as cancelled, why would we start responsiveness checks").

Do we have a solution for case where GPUP is hung before InitializeGPUProcess?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

For a GPU process that hangs before InitializeGPUProcess: the web process still
times out on its GPU process connection after 15 seconds
(WebProcess::gpuProcessTimeoutDuration) and reports it, which ends in
GPUProcessProxy::didBecomeUnresponsive() regardless of the responsiveness
checks. So such a process is terminated after 15 seconds instead of 3.


// Longer than the responsiveness timeout.
Util::runFor(4_s);
kill(gpuProcessPID, SIGCONT);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So should we add a variant of the test where we don't send this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added as GPUProcess.TerminatedWhenHungBeforeInitialization. The timeout in
WebProcess::gpuProcessTimeoutDuration() is 15 seconds, which would make the
test take about 16 seconds, so I added a testing-only preference,
WebProcessGPUProcessTimeoutForTesting, next to ChildProcessDebuggabilityEnabled
which already controls that timeout. With it the test takes under 2 seconds.

Not sure is the preference is the right approach, but without it the test would take
16 seconds. What would you recommend?

matt2e added a commit to block/buzz-app that referenced this pull request Nov 13, 2026
…3342)

## Problem

On macOS, Buzz sometimes opens to a window that never paints. It has
been seen when macOS reopens Buzz at login after a restart. The app is
otherwise alive: the page loads and its JavaScript and IPC keep running.
Only quitting and reopening Buzz clears it.

The cause is in WebKit, not in Buzz:

1. On the first page connection, WebKit's GPU process makes a blocking
audio-codec lookup. Right after login the macOS audio services it calls
are still starting, so the lookup is slow.
2. WebKit gives the GPU process 3 s to answer. When it misses that,
WebKit declares it unresponsive and asks launchd to kill it.
3. macOS refuses the kill with `Could not signal service
com.apple.WebKit.GPU: 1: Operation not permitted`. WebKit stops tracking
the process anyway and never starts a replacement.
4. The page stays connected to the abandoned process, so nothing is
painted again.

Since #2183 the main window is clear at the native level, so the failure
shows as a fully see-through window with only the traffic lights.

## What this changes

- **Solid native window outside the welcome intro.** The main `NSWindow`
gets a background that matches `--bg-app`, so a launch that never paints
shows a blank window instead of nothing. The intro clears it again and
keeps its current look.
- **Kill the GPU process WebKit abandoned.** A new macOS-only module,
`src-tauri/src/gpu_reaper.rs`, checks once a second which GPU process
the main webview reports. A process that was reported earlier, is no
longer reported for two checks, and is still the same running WebKit GPU
binary gets `SIGKILL`. WebKit then starts a new GPU process and the page
paints without a reload.
- **Skip the check while the webview has no content process.** WebKit
also reports no GPU process in that state, while other webviews keep
using the healthy one. This came out of review of the previous commit.
- **Write the watcher's messages to the unified log.** An app opened
from Finder, the Dock or at login has no stderr, so the watcher's own
lines were lost and a kill that macOS refused left nothing to find. The
four messages now also go to the unified log at notice level.

The watcher never kills a process it did not first see reported by
Buzz's own webview. It does nothing if the private selectors are
missing.

## Research

Upstream WebKit:
- [WebKit bug 324797](https://bugs.webkit.org/show_bug.cgi?id=324797):
"GPU process termination fails with EPERM after unresponsiveness,
leaving WKWebView blank". Open and unassigned. The log lines match ours.
The reporter induced the stall on purpose and has not established a
natural trigger.
- [WebKit bug 331017](https://bugs.webkit.org/show_bug.cgi?id=331017)
and its unmerged fix
[WebKit/WebKit#80415](WebKit/WebKit#80415): the
GPU process is killed as unresponsive while it is only slow to start on
a loaded machine. That fix moves the start of the checks to the end of
GPU process initialisation. Our stall happens later, on the first page
connection, so it may not cover this case.
- [WebKit commit
5e174d0](WebKit/WebKit@5e174d0)
for bug 300013: adds the Opus and Vorbis capability check that runs when
a GPU connection is created. This is the lookup our GPU process was
blocked in.

Other apps with the same failure:
- [Saqoosha/Canopy#299](Saqoosha/Canopy#299),
merged: a WKWebView pane stayed blank after the GPU process hung at
launch inside CoreAudio. The app now finds the abandoned process through
`_gpuProcessIdentifier` and kills it, after which "WebKit launch[es] a
new GPU process, and the page paints at once". This PR takes the same
approach.
-
[DigitalN/mac-engine-room#5](DigitalN/mac-engine-room#5),
merged: "After a reboot the dashboard opened at login and stayed blank."
The GPU process hung setting up audio, the kill failed with the same
message, and the app now ends that process itself.
-
[bambulab/BambuStudio#12575](bambulab/BambuStudio#12575),
open: on macOS 27.0.1 every web view is blank because the GPU process
hangs during CoreAudio initialisation. It does not mention the failed
kill.
- [yetone/magpie#3272](yetone/magpie#3272),
open: the same three log lines on macOS 27.0.1, with the GPU process
still alive afterwards. It happened mid-session, not at launch.

We found no report from Tauri, wry or tao, and no public report that
names the Opus lookup as the trigger.

## Verification

Done, on macOS 27.2:
- Eight unit tests for the watcher pass, including one that kills a
child process the test starts. Removing either safety check makes tests
fail.
- `clippy --all-targets -D warnings` is clean for the crate.
- In a standalone WKWebView test app, WebKit's own kill was refused
every time. Killing the abandoned GPU process from the app brought up a
new one and both web views painted in under half a second. Reloading the
page did not recover it.
- Hardened-runtime, ad-hoc signed release builds of Buzz were launched
with their own GPU process stopped. In five runs before the logging
commit, the watcher killed it 1.1 to 2.0 s after WebKit gave up, WebKit
started a new GPU process within half a second, and the page painted
without a reload. Two of those runs were launched hidden through
LaunchServices and never activated.
- A build with the logging commit was run once the same way, launched in
the background through LaunchServices. The watcher's `Killed WebKit GPU
process` line was in the unified log at default level 1.77 s after
WebKit gave up. launchd's kill record followed and a new GPU process was
running.
- The timer keeps its rate while the window is hidden. In a hidden,
never-activated test app, App Nap began 31 s after launch and a 1 s
timer still fired every 0.6 to 1.4 s for the next 270 s. The stall
happens within the first 5 s.
- The solid background was checked on screen in light and dark mode, and
the welcome intro still shows the desktop.
- An independent review found no way for the watcher to kill another
app's process or a reused pid. Its one warning is fixed by the third
commit.

Not done:
- The kill has not run from a Developer ID signed, notarized build.
Signed builds come only from `main`, so the restart test below runs on
the first signed build after merge.
- The watcher's `Could not kill` line has not been produced by Buzz,
because no kill was refused in any run. The same log call with a
refused-kill text was read back from a standalone hardened-runtime
binary.
- Timer behaviour on battery power and with the screen locked has not
been measured. A missed process only leaves the window blank, as it is
today.

Known limits, left for a follow-up:
- The native colour follows the macOS appearance. A user whose Buzz
theme differs can see the other colour until the page paints.
- The session that shows the welcome intro keeps a see-through native
window until the next launch.

## Test plan

- [x] Normal launch in light and dark mode: solid `--bg-app` before
first paint, usual look afterwards.
- [x] Fresh welcome intro: the desktop still shows through.
- [x] Release build, window visible and hidden: `kill -STOP` the build's
own `com.apple.WebKit.GPU` process and wait. The log shows
`GPUProcessProxy::didBecomeUnresponsive`, then the watcher's kill, then
a new GPU process, and the page paints again.
- [ ] Signed build: restart with Buzz open and confirm it reopens to a
painted window. Then check the log:
- `log show --last 30m --predicate 'process == "Buzz" AND eventMessage
CONTAINS "GPU process"'`
- `log show --last 30m --predicate 'process == "launchd" AND
eventMessage CONTAINS "exited due to SIGKILL | sent by Buzz"'`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Matt Toohey <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
https://bugs.webkit.org/show_bug.cgi?id=331017

Reviewed by NOBODY (OOPS!).

GPUProcessProxy began its responsiveness checks in didFinishLaunching(), so the
responsiveness timer armed by createGPUProcessConnection() started as soon as
the process had launched. A GPU process that needed more than three seconds to
start and to handle InitializeGPUProcess, for example on a loaded machine, was
terminated as unresponsive although it was not hung. On the Windows port
didFinishLaunching() is called right after CreateProcess() returns, so the
whole startup of the process counted against the timeout.

Begin the responsiveness checks when the GPU process replies to
InitializeGPUProcess instead, like NetworkProcessProxy does since 248341@main.
A check requested before that is delayed until the reply arrives.

The reply handler is also called when the reply is cancelled, for example when
the process exits before it has initialized. Do not begin the checks in that
case, neither in GPUProcessProxy nor in NetworkProcessProxy.

A GPU process that hangs before it replies to InitializeGPUProcess is still
terminated: a web process which waits for it times out and reports that to the
UI process. That timeout is 15 seconds. Add the
WebProcessGPUProcessTimeoutForTesting preference to override it, so that the
test of this case does not have to wait that long.

Tests: Tools/TestWebKitAPI/Tests/WebKit/WKWebView/GPUProcess.mm

* Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml:
* Source/WebKit/UIProcess/GPU/GPUProcessProxy.cpp:
(WebKit::GPUProcessProxy::GPUProcessProxy):
(WebKit::GPUProcessProxy::didFinishLaunching):
* Source/WebKit/UIProcess/Network/NetworkProcessProxy.cpp:
(WebKit::NetworkProcessProxy::sendCreationParametersToNewProcess):
* Source/WebKit/WebProcess/WebProcess.cpp:
(WebKit::WebProcess::gpuProcessTimeoutDuration const):
(WebKit::WebProcess::updateProcessPreferences):
* Source/WebKit/WebProcess/WebProcess.h:
* Tools/TestWebKitAPI/Tests/WebKit/WKWebView/GPUProcess.mm:
(-[GPUProcessLaunchObserver observeValueForKeyPath:ofObject:change:context:]):
(TestWebKitAPI::TEST(GPUProcess, NotTerminatedWhenSlowToInitialize)):
(TestWebKitAPI::TEST(GPUProcess, TerminatedWhenHungBeforeInitialization)):
@yury-s yury-s removed the merging-blocked Applied to prevent a change from being merged label Nov 13, 2026
@yury-s
yury-s force-pushed the eng/GPU-process-is-terminated-as-unresponsive-while-it-is-still-starting-up branch from 50c6551 to fe90600 Compare November 13, 2026 18:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

WebKit Process Model Bugs related to WebKit's multi-process architecture

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants