Repository navigation
Conversation
|
EWS run on previous version of this PR (hash 50c6551) Details |
| sendWithAsyncReply(Messages::GPUProcess::InitializeGPUProcess(WTF::move(parameters)), [initializationActivityAndGrant = initializationActivityAndGrant()] { }); | ||
| sendWithAsyncReply(Messages::GPUProcess::InitializeGPUProcess(WTF::move(parameters)), [weakThis = WeakPtr { *this }, initializationActivityAndGrant = initializationActivityAndGrant()] { | ||
| if (RefPtr protectedThis = weakThis) | ||
| protectedThis->beginResponsivenessChecks(); |
There was a problem hiding this comment.
For style points you could add a check if the async reply is a cancel reply (here and in network process), so that it's consistent ("if the reply is answered as cancelled, why would we start responsiveness checks").
Do we have a solution for case where GPUP is hung before InitializeGPUProcess?
There was a problem hiding this comment.
Done.
For a GPU process that hangs before InitializeGPUProcess: the web process still
times out on its GPU process connection after 15 seconds
(WebProcess::gpuProcessTimeoutDuration) and reports it, which ends in
GPUProcessProxy::didBecomeUnresponsive() regardless of the responsiveness
checks. So such a process is terminated after 15 seconds instead of 3.
|
|
||
| // Longer than the responsiveness timeout. | ||
| Util::runFor(4_s); | ||
| kill(gpuProcessPID, SIGCONT); |
There was a problem hiding this comment.
So should we add a variant of the test where we don't send this?
There was a problem hiding this comment.
Added as GPUProcess.TerminatedWhenHungBeforeInitialization. The timeout in
WebProcess::gpuProcessTimeoutDuration() is 15 seconds, which would make the
test take about 16 seconds, so I added a testing-only preference,
WebProcessGPUProcessTimeoutForTesting, next to ChildProcessDebuggabilityEnabled
which already controls that timeout. With it the test takes under 2 seconds.
Not sure is the preference is the right approach, but without it the test would take
16 seconds. What would you recommend?
…3342) ## Problem On macOS, Buzz sometimes opens to a window that never paints. It has been seen when macOS reopens Buzz at login after a restart. The app is otherwise alive: the page loads and its JavaScript and IPC keep running. Only quitting and reopening Buzz clears it. The cause is in WebKit, not in Buzz: 1. On the first page connection, WebKit's GPU process makes a blocking audio-codec lookup. Right after login the macOS audio services it calls are still starting, so the lookup is slow. 2. WebKit gives the GPU process 3 s to answer. When it misses that, WebKit declares it unresponsive and asks launchd to kill it. 3. macOS refuses the kill with `Could not signal service com.apple.WebKit.GPU: 1: Operation not permitted`. WebKit stops tracking the process anyway and never starts a replacement. 4. The page stays connected to the abandoned process, so nothing is painted again. Since #2183 the main window is clear at the native level, so the failure shows as a fully see-through window with only the traffic lights. ## What this changes - **Solid native window outside the welcome intro.** The main `NSWindow` gets a background that matches `--bg-app`, so a launch that never paints shows a blank window instead of nothing. The intro clears it again and keeps its current look. - **Kill the GPU process WebKit abandoned.** A new macOS-only module, `src-tauri/src/gpu_reaper.rs`, checks once a second which GPU process the main webview reports. A process that was reported earlier, is no longer reported for two checks, and is still the same running WebKit GPU binary gets `SIGKILL`. WebKit then starts a new GPU process and the page paints without a reload. - **Skip the check while the webview has no content process.** WebKit also reports no GPU process in that state, while other webviews keep using the healthy one. This came out of review of the previous commit. - **Write the watcher's messages to the unified log.** An app opened from Finder, the Dock or at login has no stderr, so the watcher's own lines were lost and a kill that macOS refused left nothing to find. The four messages now also go to the unified log at notice level. The watcher never kills a process it did not first see reported by Buzz's own webview. It does nothing if the private selectors are missing. ## Research Upstream WebKit: - [WebKit bug 324797](https://bugs.webkit.org/show_bug.cgi?id=324797): "GPU process termination fails with EPERM after unresponsiveness, leaving WKWebView blank". Open and unassigned. The log lines match ours. The reporter induced the stall on purpose and has not established a natural trigger. - [WebKit bug 331017](https://bugs.webkit.org/show_bug.cgi?id=331017) and its unmerged fix [WebKit/WebKit#80415](WebKit/WebKit#80415): the GPU process is killed as unresponsive while it is only slow to start on a loaded machine. That fix moves the start of the checks to the end of GPU process initialisation. Our stall happens later, on the first page connection, so it may not cover this case. - [WebKit commit 5e174d0](WebKit/WebKit@5e174d0) for bug 300013: adds the Opus and Vorbis capability check that runs when a GPU connection is created. This is the lookup our GPU process was blocked in. Other apps with the same failure: - [Saqoosha/Canopy#299](Saqoosha/Canopy#299), merged: a WKWebView pane stayed blank after the GPU process hung at launch inside CoreAudio. The app now finds the abandoned process through `_gpuProcessIdentifier` and kills it, after which "WebKit launch[es] a new GPU process, and the page paints at once". This PR takes the same approach. - [DigitalN/mac-engine-room#5](DigitalN/mac-engine-room#5), merged: "After a reboot the dashboard opened at login and stayed blank." The GPU process hung setting up audio, the kill failed with the same message, and the app now ends that process itself. - [bambulab/BambuStudio#12575](bambulab/BambuStudio#12575), open: on macOS 27.0.1 every web view is blank because the GPU process hangs during CoreAudio initialisation. It does not mention the failed kill. - [yetone/magpie#3272](yetone/magpie#3272), open: the same three log lines on macOS 27.0.1, with the GPU process still alive afterwards. It happened mid-session, not at launch. We found no report from Tauri, wry or tao, and no public report that names the Opus lookup as the trigger. ## Verification Done, on macOS 27.2: - Eight unit tests for the watcher pass, including one that kills a child process the test starts. Removing either safety check makes tests fail. - `clippy --all-targets -D warnings` is clean for the crate. - In a standalone WKWebView test app, WebKit's own kill was refused every time. Killing the abandoned GPU process from the app brought up a new one and both web views painted in under half a second. Reloading the page did not recover it. - Hardened-runtime, ad-hoc signed release builds of Buzz were launched with their own GPU process stopped. In five runs before the logging commit, the watcher killed it 1.1 to 2.0 s after WebKit gave up, WebKit started a new GPU process within half a second, and the page painted without a reload. Two of those runs were launched hidden through LaunchServices and never activated. - A build with the logging commit was run once the same way, launched in the background through LaunchServices. The watcher's `Killed WebKit GPU process` line was in the unified log at default level 1.77 s after WebKit gave up. launchd's kill record followed and a new GPU process was running. - The timer keeps its rate while the window is hidden. In a hidden, never-activated test app, App Nap began 31 s after launch and a 1 s timer still fired every 0.6 to 1.4 s for the next 270 s. The stall happens within the first 5 s. - The solid background was checked on screen in light and dark mode, and the welcome intro still shows the desktop. - An independent review found no way for the watcher to kill another app's process or a reused pid. Its one warning is fixed by the third commit. Not done: - The kill has not run from a Developer ID signed, notarized build. Signed builds come only from `main`, so the restart test below runs on the first signed build after merge. - The watcher's `Could not kill` line has not been produced by Buzz, because no kill was refused in any run. The same log call with a refused-kill text was read back from a standalone hardened-runtime binary. - Timer behaviour on battery power and with the screen locked has not been measured. A missed process only leaves the window blank, as it is today. Known limits, left for a follow-up: - The native colour follows the macOS appearance. A user whose Buzz theme differs can see the other colour until the page paints. - The session that shows the welcome intro keeps a see-through native window until the next launch. ## Test plan - [x] Normal launch in light and dark mode: solid `--bg-app` before first paint, usual look afterwards. - [x] Fresh welcome intro: the desktop still shows through. - [x] Release build, window visible and hidden: `kill -STOP` the build's own `com.apple.WebKit.GPU` process and wait. The log shows `GPUProcessProxy::didBecomeUnresponsive`, then the watcher's kill, then a new GPU process, and the page paints again. - [ ] Signed build: restart with Buzz open and confirm it reopens to a painted window. Then check the log: - `log show --last 30m --predicate 'process == "Buzz" AND eventMessage CONTAINS "GPU process"'` - `log show --last 30m --predicate 'process == "launchd" AND eventMessage CONTAINS "exited due to SIGKILL | sent by Buzz"'` 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Matt Toohey <[email protected]> Co-authored-by: Claude Opus 5.5 <[email protected]>
https://bugs.webkit.org/show_bug.cgi?id=331017 Reviewed by NOBODY (OOPS!). GPUProcessProxy began its responsiveness checks in didFinishLaunching(), so the responsiveness timer armed by createGPUProcessConnection() started as soon as the process had launched. A GPU process that needed more than three seconds to start and to handle InitializeGPUProcess, for example on a loaded machine, was terminated as unresponsive although it was not hung. On the Windows port didFinishLaunching() is called right after CreateProcess() returns, so the whole startup of the process counted against the timeout. Begin the responsiveness checks when the GPU process replies to InitializeGPUProcess instead, like NetworkProcessProxy does since 248341@main. A check requested before that is delayed until the reply arrives. The reply handler is also called when the reply is cancelled, for example when the process exits before it has initialized. Do not begin the checks in that case, neither in GPUProcessProxy nor in NetworkProcessProxy. A GPU process that hangs before it replies to InitializeGPUProcess is still terminated: a web process which waits for it times out and reports that to the UI process. That timeout is 15 seconds. Add the WebProcessGPUProcessTimeoutForTesting preference to override it, so that the test of this case does not have to wait that long. Tests: Tools/TestWebKitAPI/Tests/WebKit/WKWebView/GPUProcess.mm * Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml: * Source/WebKit/UIProcess/GPU/GPUProcessProxy.cpp: (WebKit::GPUProcessProxy::GPUProcessProxy): (WebKit::GPUProcessProxy::didFinishLaunching): * Source/WebKit/UIProcess/Network/NetworkProcessProxy.cpp: (WebKit::NetworkProcessProxy::sendCreationParametersToNewProcess): * Source/WebKit/WebProcess/WebProcess.cpp: (WebKit::WebProcess::gpuProcessTimeoutDuration const): (WebKit::WebProcess::updateProcessPreferences): * Source/WebKit/WebProcess/WebProcess.h: * Tools/TestWebKitAPI/Tests/WebKit/WKWebView/GPUProcess.mm: (-[GPUProcessLaunchObserver observeValueForKeyPath:ofObject:change:context:]): (TestWebKitAPI::TEST(GPUProcess, NotTerminatedWhenSlowToInitialize)): (TestWebKitAPI::TEST(GPUProcess, TerminatedWhenHungBeforeInitialization)):
50c6551 to
fe90600
Compare
|
EWS run on current version of this PR (hash fe90600) Details |
fe90600
fe90600