You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Every macOS build in the merge queue compiles the native library from scratch, which takes about 31 minutes on macos-14. pr_build_macos.yml saves its cargo cache only when github.ref == 'refs/heads/main', but the macOS workflow runs only in the merge queue and on pull requests with the run-macos-tests label, never on main. So the entry is never written and every restore misses.
Today the macOS path (31 minute native build plus the 39 minute [expressions] suites, about 70 minutes) is shorter than the Spark SQL path, but once the Spark SQL shards are rebalanced (#6388) it becomes the longest path in a queue run.
Describe the potential solution
Give the macOS build the same push-to-main cache refresh that the Linux build has:
a build_macos_full POLICY output next to build_macos, the way build_linux_full sits next to build_linux, with build_macos gaining the push tier
a cache-refresh-only input on pr_build_macos.yml that skips the test matrix on push
in that mode, build-native checks for the entry with lookup-only and builds and saves only when the key is missing
Key the entry on Cargo.lock and Cargo.toml only, as #6387 does for Linux. The push run then builds once per dependency change and otherwise finishes in about a minute.
Additional context
This should land after #6387. The macOS entry adds about 1.4 GB, and the cache budget has no room for it until #6387 frees some.
Activity
ci: keep a macOS native build cache on main (apache#6395)