Repository navigation
[macOS] macos-15 (x86) runner is excruciatingly slow #12545
Description
Description
I have recently noticed that the x86 macos-15 runner has become uselessly slow. I have no idea whether it's overloaded, or if it's a regression, but it is incapable of compiling software at a normal rate.
Platforms affected
- Azure DevOps
- GitHub Actions - Standard Runners
- GitHub Actions - Larger Runners
Runner images affected
- Ubuntu 22.04
- Ubuntu 24.04
- macOS 13
- macOS 13 Arm64
- macOS 14
- macOS 14 Arm64
- macOS 15
- macOS 15 Arm64
- Windows Server 2019
- Windows Server 2022
- Windows Server 2025
Image version and build link
Current runner version: '2.325.0'
Runner Image Provisioner
Hosted Compute Agent
Version: 20250703.357
Commit: 07daf62238a21140d93e045a38f3784d75c509e1
Build Date: 2025-07-03T14:39:09Z
Operating System
macOS
15.5
24F74
Runner Image
Image: macos-15-arm64
Version: 20250630.1914
Included Software: https://github.com/actions/runner-images/blob/macos-15-arm64/20250630.1914/images/macos/macos-15-arm64-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/macos-15-arm64%2F20250630.1914
Is it regression?
Maybe
Expected behavior
The runner should be able to compile more than 400 translation units over 5 hours
Actual behavior
The runner is incapable of compiling more than 400 units into objects over 5 hours. Previously, it was able to compile thousands (>6000) objects per hour.
Repro steps
- Try to compile a larger project
- Fail spectacularly
Activity
[-][macOS] macos-15 runner is excruciatingly slow[/-][+][macOS] macos-15 (x86) runner is excruciatingly slow[/+]ijunaidm1 commentedon Jul 8, 2025
Hi @dumbmoron - We will start investigating on this issue and will update you accordingly.
amlcurran commentedon Jul 9, 2025
Can confirm that
Image: macos-15-arm64
Version: 20250630.1914
Is also running much slower for us than previous, similar to my previously reported issues #11509
sangeeths03 commentedon Jul 11, 2025
Hi @dumbmoron, thanks for the detailed report.
We've tested this thoroughly on our side using a synthetic workload that compiles over 6000 translation units into a single executable. This benchmark was executed across macos-13-arm64, macos-14-arm64, and macos-15-arm64 runners using the latest image version , and we did not observe any significant slowdown on macos-15-arm64 compared to earlier versions.
System conditions also appear normal:
~35 GiB of free disk space (df -h)
Build times are within expected ranges
Full run log: - https://github.com/sangeeths03/runner-images-sangeeth/actions/runs/16216838686
Given these results, we’re currently unable to reproduce the performance issue on our side.
As a next step, we'll fork your repository and attempt to reproduce the problem using your exact build and configuration. This should help us identify whether the slowdown is related to specific build scripts, link step behavior, or other project-specific factors.
Additionally, a new macOS 15 ARM runner image is scheduled for release this week. Once it's live, we recommend trying your build again to see if the issue still persists under the updated environment.
We'll follow up after deeper testing with your repo.
Also, could you please confirm:
Thanks again for your patience as we investigate this further.
dumbmoron commentedon Jul 11, 2025
Hi @sangeeths03,
Thank you for looking into this!
The build normally takes around 20 hours to complete (4x 5 hour build jobs). Recently, I noticed that it can take easily up to 50 hours (10x 5 hour build jobs).
The slowness does seem to be intermittent - during some builds, the runner builds at a normal pace (maybe 1 object file per second, more or less, or much faster if it's using
ccached objects). In other builds, the pipeline slows to a crawl, producing 1 object file per 10 to 20 minutes.We currently have a x86 pipeline experiencing this: https://github.com/imputnet/helium-macos/actions/runs/16207265303/job/45787787964
It has been running for 2 hours so far, and only managed to produce 259 object files (normally, it is able to compile around 1200-1600 units per hour).
sangeeths03 commentedon Jul 17, 2025
Hi @dumbmoron,
Thanks again for your detailed reporting and for your patience throughout the investigation.
We've been monitoring your recent workflow runs, and we’re happy to see that the build durations now appear to be back within the expected range (around 20 hours total). This aligns with the performance observed prior to the slowdown.
Additionally, based on our internal testing and environment health checks, we haven’t been able to reproduce the degradation on the macos-15 runners. No infrastructure-level regressions have been identified so far.
Given this, could you please confirm:
Are your builds consistently completing within the expected duration now?
Have you observed any further intermittent slowdowns recently?
If everything is running smoothly again, we’ll go ahead and consider this issue resolved. Otherwise, happy to dig deeper with more specific job links or details.
Thanks again!
dumbmoron commentedon Jul 18, 2025
Hi @sangeeths03,
The arm64 builds are more or less fine, and the x86 builds are still intermittently slow.
Yes, we currently have a running job that compiled only 500 units in 4 hours:
https://github.com/imputnet/helium-macos/actions/runs/16352765637/job/46232654075
badredaha commentedon Jul 30, 2025
+1
badredaha commentedon Jul 30, 2025
Hi,
I use AzureDevops pipeline and My iOS build app take more than 2 hours! and after that the build failed!
Any suggestions ?
sangeeths03 commentedon Aug 6, 2025
Hi @dumbmoron ,A new image was recently released for macOS 15 Intel (x86). Could you please confirm if the slowness reported still persists with the updated image?
26 remaining items
jpfeuffer commentedon Oct 1, 2025
We have exactly the same issue for a c++ project (using default Xcode 16) Both the now deprecated macOS-13 (intel) as well as macOS-26 are 5 times faster than this particular image.
It takes 1 hour for a FULLY CCACHED build. On both other runners this is less around 10 mins.
https://github.com/OpenMS/OpenMS/actions/workflows/openms_ci_matrix_full.yml
tmiw commentedon Oct 9, 2025
I wonder if our project may be running into a similar problem. I recently switched our repo to macos-15-intel and have noticed that the ARM based runners are passing pretty consistently while the Intel one is fairly consistently failing. Example run. Unfortunately I'm fairly sure it's not a bug as the tests pass on my 2019 MBP.
Anyway, hope this helps debug the problem!
jonaski commentedon Nov 11, 2025
macos-13 is currently failing with
This is a scheduled macos-13 brownout. The macOS-13 based runner images are being deprecated. For more details, see https://github.com/actions/runner-images/issues/13046.But macos-15-intel is too slow to be usable.
ci: switch to macos-13
Update macOS version in CI workflow
cclauss commentedon Dec 12, 2025
Can
macos-13vs.macos-15-intelperformance differences be documented with these reports?Run CI on macos-14 arm64 image because it is faster than on macos-15-…
chore(ci): upgrade macos runners from macos-15 to macos-26
feat(macos): add Intel (x86_64) build with separate DMG
ci(release): switch macOS Intel runner from macos-13 to macos-15 (#42)