Repository navigation
[sanitizer_common] [Darwin] Adopt _dyld_get_dyld_header - #182943
Merged
Merged
Conversation
Member
|
@llvm/pr-subscribers-compiler-rt-sanitizer Author: Andrew Haberlandt (ndrewh) ChangesThere is an issue on recent macOS versions with rdar://167854578 3 Files Affected:
diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_procmaps_mac.cpp b/compiler-rt/lib/sanitizer_common/sanitizer_procmaps_mac.cpp
index 979729f15aa16..0e80f2e3916b2 100644
--- a/compiler-rt/lib/sanitizer_common/sanitizer_procmaps_mac.cpp
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_procmaps_mac.cpp
@@ -176,7 +176,7 @@ void MemoryMappingLayout::Reset() {
// The dyld load address should be unchanged throughout process execution,
// and it is expensive to compute once many libraries have been loaded,
// so cache it here and do not reset.
-static mach_header *dyld_hdr = 0;
+static const mach_header *dyld_hdr = 0;
static const char kDyldPath[] = "/usr/lib/dyld";
static const int kDyldImageIdx = -1;
@@ -244,14 +244,18 @@ extern intptr_t _dyld_get_image_slide(const struct mach_header* mh);
extern int dyld_shared_cache_iterate_text(
const uuid_t cacheUuid,
void (^callback)(const dyld_shared_cache_dylib_text_info *info));
+SANITIZER_WEAK_IMPORT const struct mach_header *_dyld_get_dyld_header(void);
} // extern "C"
-static mach_header *GetDyldImageHeaderViaSharedCache() {
+static const mach_header *GetDyldImageHeaderViaSharedCache() {
uuid_t uuid;
bool hasCache = _dyld_get_shared_cache_uuid(uuid);
if (!hasCache)
return nullptr;
+ if (&_dyld_get_dyld_header != nullptr)
+ return _dyld_get_dyld_header();
+
size_t cacheLength;
__block uptr cacheStart = (uptr)_dyld_get_shared_cache_range(&cacheLength);
CHECK(cacheStart && cacheLength);
diff --git a/compiler-rt/lib/sanitizer_common/weak_symbols.txt b/compiler-rt/lib/sanitizer_common/weak_symbols.txt
index 77e7b5d9f702e..600ab8a7c649c 100644
--- a/compiler-rt/lib/sanitizer_common/weak_symbols.txt
+++ b/compiler-rt/lib/sanitizer_common/weak_symbols.txt
@@ -10,3 +10,4 @@ ___sanitizer_symbolize_demangle
___sanitizer_symbolize_flush
___sanitizer_symbolize_set_demangle
___sanitizer_symbolize_set_inline_frames
+__dyld_get_dyld_header
diff --git a/compiler-rt/lib/tsan/go/buildgo.sh b/compiler-rt/lib/tsan/go/buildgo.sh
index d9e56402ad48f..1340071819fcb 100755
--- a/compiler-rt/lib/tsan/go/buildgo.sh
+++ b/compiler-rt/lib/tsan/go/buildgo.sh
@@ -165,7 +165,7 @@ elif [ "$GOOS" = "netbsd" ]; then
"
elif [ "$GOOS" = "darwin" ]; then
OSCFLAGS="-fPIC -Wno-unused-const-variable -Wno-unknown-warning-option -mmacosx-version-min=10.7"
- OSLDFLAGS="-lpthread -fPIC -fpie -mmacosx-version-min=10.7"
+ OSLDFLAGS="-lpthread -fPIC -fpie -mmacosx-version-min=10.7 -Wl,-U,__dyld_get_dyld_header"
SRCS="
$SRCS
../rtl/tsan_platform_mac.cpp
|
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
ndrewh
force-pushed
the
GetDyldImageHeaderViaSharedCache
branch
from
February 23, 2026 21:44
ea41f5f to
bdf57bf
Compare
Contributor
Author
|
Fixed clang-format |
DanBlackwell
approved these changes
Feb 23, 2026
gopherbot
pushed a commit
to golang/go
that referenced
this pull request
Mar 19, 2026
With llvm/llvm-project#182943, the race detector syso has a weak import of __dyld_get_dyld_header, which is only defined on newer macOS (26.4+). For external linking with a pre-Xcode 26.4 C toolchain, we need to tell the C linker to permit that symbol not being defined. Pass a flag to do so. Change-Id: I95a3cd2c7fd3ad50bc47985b3ecca0d4e8352162 Reviewed-on: https://go-review.googlesource.com/c/go/+/755261 LUCI-TryBot-Result: Go LUCI <[email protected]> Reviewed-by: David Chase <[email protected]> Reviewed-by: Lasse Folger <[email protected]>
Contributor
Author
|
/cherry-pick 2e7d07a |
Contributor
Author
|
I think we should cherry-pick this to make sure that clang 22 supports ASAN on macOS 26.4. |
Member
Error: Command failed due to missing milestone. |
Contributor
Author
|
/cherry-pick 2e7d07a |
Member
|
/pull-request #188913 |
dylan-conway
added a commit
to oven-sh/bun
that referenced
this pull request
Mar 30, 2026
…t mode DirectBuild: deps simple enough to list files compile as first-class ninja edges instead of a cmake sub-process. tinycc converted — drops the overlay CMakeLists.txt and the recurring ASAN workarounds on the c2str host tool, which now gets -fno-sanitize=all unconditionally. ASAN dyld shim: macOS 26.4's Dyld.framework reimplemented dyld_shared_cache_iterate_text in Swift with a _Block_copy that deadlocks ASAN init (llvm/llvm-project#182943). Shim interposes a non-allocating version using _dyld_get_dyld_header. Linked via LC_LOAD_DYLIB + @rpath so it auto-loads with bun-debug — no env var. Darwin+ASAN only. workarounds.ts: self-obsoleting registry. Every temporary fix registers an expectedToBeFixed predicate; configure fails with cleanup instructions once the upstream fix ships. build.ts: quiet build output when exec args present (original bd behavior), `--` separator for flag disambiguation, `--target` accepts space-separated value, colored target name in done message, signal exit re-raised. Arg routing documented in file header. tty.ts: centralized ANSI/TTY abstraction. nameColor() hashes names to stable colors; stream.ts and the done message share it. config.ts/tools.ts: clangVersion captured from the existing toolchain-resolve --version spawn, exposed on Config. findTool returns {path, version}. Also: - configure() no longer prints; build.ts decides based on quiet mode - provides.sources deps get phonies pointing at compiled .o files - scripts/bd, scripts/bd.ps1 removed (bd is a package.json script) - scripts/build/CLAUDE.md: architecture doc with goals, ninja primer, common tasks, arg routing
c-rhodes
pushed a commit
to llvmbot/llvm-project
that referenced
this pull request
Mar 31, 2026
(cherry picked from commit 2e7d07a)
ndrewh
referenced
this pull request
in modular/modular
Apr 12, 2026
past the ASAN runtime fix (llvm/llvm-project#191039). MODULAR_ORIG_COMMIT_REV_ID: 1773da5eabdc1456e1fac443ccf0aac439c6a895
dylan-conway
pushed a commit
to oven-sh/bun
that referenced
this pull request
Jul 17, 2026
…macOS (#34508) ## Problem `bun bd` aborts at startup on macOS releases older than 26.4 (seen on 15.6.1, Homebrew llvm@21): ``` AddressSanitizer: CHECK failed: sanitizer_procmaps_mac.cpp:214 "((res)) == ((0))" (0xffffffffffffffff, 0x0) <empty stack> Abort trap: 6 ``` The post-link `bun-debug --revision` smoke test dies with this, so the debug build never completes. Workaround has been `bun run build --asan=off`. ## Cause `scripts/build/shims/asan-dyld-shim.c` interposes `dyld_shared_cache_iterate_text` to keep ASAN init from deadlocking on macOS 26.4 (llvm/llvm-project#182943). It looks up the private `_dyld_get_dyld_header` via `dlsym` to synthesize the one cache entry ASAN needs. Apple's own `dyld_priv.h` marks that symbol ["Added in macOS/iOS 26.4"](https://github.com/apple-oss-distributions/dyld/blob/dyld-1376.6/include/mach-o/dyld_priv.h) (first appears in dyld-1376.6; absent from dyld-1340 and every earlier tag). On any macOS < 26.4, `dlsym` returns `NULL`, the shim returns `-1`, and compiler-rt's `CHECK_EQ(res, 0)` aborts. The shim is linked into every `cfg.darwin && cfg.asan` build with no runtime check, so it breaks every debug build on macOS hosts that predate 26.4. ## Fix When the shim can't synthesize the entry (`dlsym` returned `NULL`, or no shared cache), delegate to the real `dyld_shared_cache_iterate_text` instead of returning `-1`. The deadlock and the getter were introduced together in 26.4, so "getter absent" is exactly "real iterate is safe". This is the same shape as the upstream compiler-rt fix (llvm/llvm-project#188913): weak-import `_dyld_get_dyld_header`, use it when present, otherwise fall through to the existing iterate path. Calling the original by name from inside the interposer does not recurse. dyld explicitly adds an identity-mapping tuple for the defining image; from [DyldRuntimeState.cpp](https://github.com/apple-oss-distributions/dyld/blob/dyld-1378/dyld/DyldRuntimeState.cpp): > `// now add specific interpose so that the generic is not applied to the interposing dylib, so it can call through to old impl` (This is also how Apple's own `DYLD_INTERPOSE` wrapper example in [dyld-interposing.h](https://github.com/apple-oss-distributions/dyld/blob/main/include/mach-o/dyld-interposing.h) works.) ## Verification Shim is only compiled for `darwin && asan` (no CI lane exists for that combination); verified the change compiles clean with `-Wall -Wextra -fblocks` against stubbed darwin headers. Behaviour on 26.4+ is unchanged: `dlsym` finds the symbol there and the same synthesize path runs as before. ## Why this is the right fix The alternative is gating shim emission on host macOS version in `shims.ts`, but the runtime check is strictly better: one binary works on both, and it matches upstream exactly. The shim is temporary regardless (self-obsoletes via `workarounds.ts` once LLVM 22.1.4+ is the floor; #34299 deletes it as part of the LLVM 22 bump), so the goal here is just to stop breaking pre-26.4 macOS until that lands. <!-- robobun:evidence:begin --> --- **no test proof** · iteration 0 · build/CI scripts only; test-proof not applicable <!-- robobun:evidence:end -->
liooil
pushed a commit
to liooil/poly
that referenced
this pull request
Aug 7, 2026
…macOS (#34508)
## Problem
`bun bd` aborts at startup on macOS releases older than 26.4 (seen on
15.6.1, Homebrew llvm@21):
```
AddressSanitizer: CHECK failed: sanitizer_procmaps_mac.cpp:214 "((res)) == ((0))" (0xffffffffffffffff, 0x0)
<empty stack>
Abort trap: 6
```
The post-link `bun-debug --revision` smoke test dies with this, so the
debug build never completes. Workaround has been `bun run build
--asan=off`.
## Cause
`scripts/build/shims/asan-dyld-shim.c` interposes
`dyld_shared_cache_iterate_text` to keep ASAN init from deadlocking on
macOS 26.4 (llvm/llvm-project#182943). It looks up the private
`_dyld_get_dyld_header` via `dlsym` to synthesize the one cache entry
ASAN needs.
Apple's own `dyld_priv.h` marks that symbol ["Added in macOS/iOS
26.4"](https://github.com/apple-oss-distributions/dyld/blob/dyld-1376.6/include/mach-o/dyld_priv.h)
(first appears in dyld-1376.6; absent from dyld-1340 and every earlier
tag). On any macOS < 26.4, `dlsym` returns `NULL`, the shim returns
`-1`, and compiler-rt's `CHECK_EQ(res, 0)` aborts.
The shim is linked into every `cfg.darwin && cfg.asan` build with no
runtime check, so it breaks every debug build on macOS hosts that
predate 26.4.
## Fix
When the shim can't synthesize the entry (`dlsym` returned `NULL`, or no
shared cache), delegate to the real `dyld_shared_cache_iterate_text`
instead of returning `-1`. The deadlock and the getter were introduced
together in 26.4, so "getter absent" is exactly "real iterate is safe".
This is the same shape as the upstream compiler-rt fix
(llvm/llvm-project#188913): weak-import `_dyld_get_dyld_header`, use it
when present, otherwise fall through to the existing iterate path.
Calling the original by name from inside the interposer does not
recurse. dyld explicitly adds an identity-mapping tuple for the defining
image; from
[DyldRuntimeState.cpp](https://github.com/apple-oss-distributions/dyld/blob/dyld-1378/dyld/DyldRuntimeState.cpp):
> `// now add specific interpose so that the generic is not applied to
the interposing dylib, so it can call through to old impl`
(This is also how Apple's own `DYLD_INTERPOSE` wrapper example in
[dyld-interposing.h](https://github.com/apple-oss-distributions/dyld/blob/main/include/mach-o/dyld-interposing.h)
works.)
## Verification
Shim is only compiled for `darwin && asan` (no CI lane exists for that
combination); verified the change compiles clean with `-Wall -Wextra
-fblocks` against stubbed darwin headers. Behaviour on 26.4+ is
unchanged: `dlsym` finds the symbol there and the same synthesize path
runs as before.
## Why this is the right fix
The alternative is gating shim emission on host macOS version in
`shims.ts`, but the runtime check is strictly better: one binary works
on both, and it matches upstream exactly. The shim is temporary
regardless (self-obsoletes via `workarounds.ts` once LLVM 22.1.4+ is the
floor; #34299 deletes it as part of the LLVM 22 bump), so the goal here
is just to stop breaking pre-26.4 macOS until that lands.
<!-- robobun:evidence:begin -->
---
**no test proof** · iteration 0 · build/CI scripts only; test-proof not
applicable
<!-- robobun:evidence:end -->
helly25
added a commit
to bazel-contrib/toolchains_llvm
that referenced
this pull request
Aug 16, 2026
Fixes #826. ## Summary Wire rules_cc's stock `lsan` feature into the same augmentation path as asan, ubsan, and tsan: - add `//toolchain/config:use_lsan` and include it in `use_common_sanitizer` - add the Darwin `libclang_rt.lsan_osx_dynamic.dylib` filegroup and route it through `dynamic_runtime_lib` - add an end-to-end `//:lsan_test` and run it with the Linux sanitizer tests - add a focused macOS CI job using LLVM 22.1.8 The Darwin version pin is deliberate. On current macOS, LLVM 19.1.7, 20.1.8, and 21.1.8 all run `main` and then deadlock during LSan's exit check. The leak-check thread calls `dyld_shared_cache_iterate_text`, which lazily loads dyld introspection code and allocates while LSan holds its allocator lock. LLVM compiler-rt #182943 switched this path to `_dyld_get_dyld_header`; that fix was backported to LLVM 22, and LLVM 22.1.8 exits normally. The configured action graph now contains a `SolibSymlink` whose declared input is `libclang_rt.lsan_osx_dynamic.dylib`, so the runtime is provisioned rather than merely found incidentally in the local execroot. ## Verification On macOS arm64: - LLVM 19.1.7: times out after completing `main` - LLVM 20.1.8: times out after completing `main` - LLVM 21.1.8: times out after completing `main` - LLVM 22.1.8: `//:lsan_test` passes - `//:sanitizer_combo_flags_test` passes - `actionlint` and `git diff --check` pass Upstream fix: llvm/llvm-project#182943
yiguolei
pushed a commit
to apache/doris
that referenced
this pull request
Sep 30, 2026
…OS 26.4+ (#68595) ### What problem does this PR solve? Issue Number: N/A Related PR: apache/doris-website#4180 (the macOS build guide change that goes with this PR) Problem Summary: **Context.** On macOS the only entry point for the BE unit tests is `run-be-ut.sh`, and it defaults to `BUILD_TYPE_UT=ASAN`. Since macOS 26.4 that default cannot start a single process: every binary linked against llvm.org's `libclang_rt.asan_osx_dynamic` deadlocks during runtime initialisation, before `main()`. `env.sh` selects that same toolchain on macOS (`llvm@20`), so a developer who follows the documented setup gets a build that hangs rather than one that fails. How the macOS toolchain gets chosen matters for the rest of this description: - **Selection.** `env.sh` writes `custom_env_mac.sh`, which puts `$(brew --prefix)/opt/<formula>/bin` on `PATH` for every formula in its `CELLARS` list. Unless `DORIS_CLANG_HOME` is already set, `env.sh` then takes the first `clang` on `PATH` and derives `CC`/`CXX` from it. - **Installation.** `CELLARS` only selects; it installs nothing. Developers install Homebrew formulae by following the macOS build guide, and each CI workflow installs from its own `brew install` list. - **Third-party libraries.** `thirdparty/installed` comes from one of two places: - the published `doris-thirdparty-prebuilt-darwin-*.tar.xz`, which `apache/doris-thirdparty`'s `build-target.yml` builds by running `thirdparty/build-thirdparty.sh` from apache/doris master; - a local run of that same script. `build.sh` also starts one by itself when `thirdparty/installed` lacks a library it checks for. **1. The problem, and what it cost** *The deadlock.* `sample` on the stalled process shows the whole chain: ``` __malloc_init (libsystem_malloc) <- early libSystem init calls malloc_default_zone() `- wrap_malloc_default_zone (asan runtime) `- AsanInitFromRtl -> AsanInitInternal -> InitializeShadowMemory `- MemoryRangeIsAvailable -> MemoryMappingLayout::Next -> get_dyld_hdr() `- dyld_shared_cache_iterate_text_swift <- macOS 26.4 reimplemented this in Swift `- _Block_copy -> malloc `- __sanitizer_mz_malloc (asan's own malloc) `- AsanInitFromRtl() <- re-enters init `- StaticSpinMutex::LockSlow <- spins on the lock it already holds ``` The cause is an OS-side change, not a Doris one. It also does not mean ASAN is broken on macOS 26: Apple's own clang sanitizer runtime runs fine on the same host, and only llvm.org's compiler-rt is affected. Upstream fixed it by weak-importing `_dyld_get_dyld_header` and using it, when present, instead of walking the shared cache (llvm/llvm-project#182943, main `2e7d07a`; backport #188913, release/22.x `7b6514c`). **The fix shipped only in 22.1.8.** 20.1.8 is the last 20.x release and 21.1.8 the last 21.x, so no version of `llvm@20` can ever be made to work. *Why it looks like a hang and not an error.* - `be/CMakeLists.txt` reaches `storage/index/ann` unconditionally, and that directory's `cmake-protect` target calls `add_subdirectory()` on `contrib/openblas`. - OpenBLAS runs an instrumented `getarch` probe from its **configure** step (`contrib/openblas/cmake/prebuild.cmake:1513`, `execute_process(COMMAND .../getarch 0 ...)`), and `execute_process` has no timeout. - So every ASAN build stops at `-- Running getarch` and never moves again. It sits at about 90% CPU and prints no diagnostic. `run-be-ut.sh` on macOS was therefore unusable at its default setting, and the failure gave the developer nothing to act on. *The second obstacle, once the toolchain is bumped.* - clang 22 added `-Wc2y-extensions` and folds it into `-Wpedantic`. - `__COUNTER__` only reached the C standard in C2y (WG14 N3457). `be/src/runtime/runtime_profile.h:73-85` uses it to give two `SCOPED_TIMER` / `SCOPED_RAW_TIMER` expansions on the same line distinct names. - As a result, a clang 22 build of any TU that includes that header fails under `-Werror`. Three representative unity TUs were enough to hit it, so the failure is not confined to one module. *The third obstacle: clang 22 cannot build the third-party libraries.* The first revision of this PR moved every macOS toolchain reference to `llvm@22`, including the third-party build. CI failed both macOS third-party jobs (7m49s, 6m23s) on the first library that exercises a new clang 22 error: ``` unixODBC 2.3.7 src/SQLBrowseConnectW.c:424:82: error: incompatible pointer types passing 'SQLSMALLINT *' (aka 'short *') to parameter of type 'int *' [-Wincompatible-pointer-types] ``` clang 16, clang 20 and Apple clang 21 report this as a warning; clang 22 makes it an error. unixODBC is only the *first* failure: the build aborts there, so every library after it is untested against clang 22. The macOS third-party libraries therefore have to stay on `llvm@20` while the BE moves to `llvm@22`. Leaving the third-party build alone does not achieve that, because it sources `env.sh`: - With `CELLARS` naming `llvm@22`, a machine without `llvm@22` gets a non-existent directory on `PATH`. - `command -v clang` then falls through to `/usr/bin/clang` (Apple clang), which is worse than either explicit choice. - The job that publishes the prebuilt archive in `apache/doris-thirdparty` is exactly such a machine: it installs only `llvm@20` and sets no `DORIS_CLANG_HOME`. *Two latent defects clang 22 then surfaced.* The bump is not a pure version change. clang 22's stricter diagnostics stop the build on two pre-existing bugs, and both are worth fixing on their own merits. | # | Site | Diagnostic | What is actually wrong | |---|---|---|---| | 1 | `be/src/load/group_commit/wal/wal_dirs_info.cpp:98` | `-Wunused-result` | `LOG(INFO) << "… err: {}", e.what();` — the `,` is the comma operator, not an argument separator, so the statement is `(LOG(INFO) << "…{}") , (e.what())`. The `{}` is never substituted, and `e.what()` is evaluated and then discarded: **the error message has never been logged**, only the literal `{}`. A repo-wide scan for the same shape finds exactly this one site (`cloud/src/common/logging.h` matches only inside macro definitions and is not a bug). | | 2 | `common/cpp/sync_point.cpp:208,210` | `-Wthread-safety-analysis` | The function holds `std::unique_lock lock(mutex_)`, then releases and re-takes the mutex with a raw `mutex_.unlock()` / `mutex_.lock()` pair around the callback, which bypasses the lock's ownership tracking. If the callback throws, the re-lock is skipped while `~unique_lock` still believes it owns the mutex, so the destructor unlocks a mutex that is not held (UB) and `num_callbacks_running_` is never decremented. `lock.unlock()` / `lock.lock()` still runs the callback unlocked, as intended, and keeps the ownership state correct on every path. | **2. What this PR does, and why it helps** - **`env.sh`**: the macOS `CELLARS` list moves from `llvm@20` to `llvm@22`, with a comment recording that 22.1.8 is the minimum and why. This is the single line that decides which clang a macOS developer builds the BE with. - **`thirdparty/build-thirdparty.sh`**: on Darwin, right after sourcing `env.sh` and next to the existing environment sanitization, the script unconditionally points `DORIS_CLANG_HOME`, `CC` and `CXX` at `$(brew --prefix llvm@20)` and puts its `bin/` first on `PATH`. If `llvm@20` is not installed, it stops with a `brew install llvm@20` hint rather than falling back to another compiler. - The override has to come after `env.sh` and has to be unconditional. `env.sh` sources `custom_env.sh`, which may export `DORIS_CLANG_HOME` for the BE, and `build.sh` has already exported the BE's `llvm@22` values before it starts this script. - Every macOS third-party build runs this script: `build.sh`'s automatic rebuild, a manual run, the pull request check in `build-thirdparty.yml`, and the `apache/doris-thirdparty` job that publishes `doris-thirdparty-prebuilt-darwin-*.tar.xz`. This one place therefore keeps all of them, and the published archive, on `llvm@20`. - `.github/workflows/build-thirdparty.yml` is unchanged: its macOS jobs already install `llvm@20`. - **`be/CMakeLists.txt`**: add `-Wno-c2y-extensions` for clang 19 and newer, in its own `add_compile_options` call **after** `-Wpedantic`. - The position matters: a later `-Wpedantic` turns the group back on. That is also why passing the flag through `EXTRA_CXX_FLAGS` does not work: that variable lands near the front of the command line. - The version gate matters because the `c2y-extensions` group only exists from clang 19 on. clang rejects an unknown `-Wno-` option like any other unknown warning option (`-Wunknown-warning-option`, fatal under `-Werror`), and this file accepts clang 16 and newer. - For clang 19 and newer the command line is exactly what it was. - **`.github/workflows/be-ut-mac.yml`**: install `llvm@22` for the BE and keep `llvm@20` next to it. - The job builds the BE on top of the downloaded darwin-arm64 prebuilt. When that archive lacks a library `build.sh` checks for, `build.sh` rebuilds the third-party libraries from source, and that rebuild now needs `llvm@20`. The archive falls behind like this whenever master changes the set of checked libraries before the next archive is published. - Both formulae are keg-only, so installing both does not change which one the BE uses. - **`be/src/load/group_commit/wal/wal_dirs_info.cpp` and `common/cpp/sync_point.cpp`**: the two fixes from the table above. Each is one line, the smallest change that removes the defect rather than suppressing the warning. No `-Wno-unused-result` / `-Wno-thread-safety-analysis` is added, because those diagnostics point at real bugs and should keep firing. - **apache/doris-website#4180**: the macOS build guide installs `llvm@22` and `llvm@20` instead of only `llvm@20`. Without it, a developer who follows the guide after this PR has no `llvm@22`, and the BE silently builds with Apple clang. What this PR deliberately does not do is add `llvm@20` to `env.sh`'s `CELLARS`: - `CELLARS` installs nothing. - `build-thirdparty.sh` does not use `PATH` to find `llvm@20`. - The `CELLARS` loop prepends each entry to `PATH`. An `llvm@20` entry after `llvm@22` would put the BE back on `llvm@20` and bring the deadlock back. **3. The classes, and how they call each other** ``` env.sh CELLARS := llvm@22 (selects a compiler; installs nothing) |- generates custom_env_mac.sh: $(brew --prefix)/opt/<cellar>/bin prepended to PATH |- DORIS_CLANG_HOME := dirname($(command -v clang))/.. -> CC / CXX / ASAN_SYMBOLIZER_PATH '- is sourced by: |- build.sh, run-be-ut.sh -> BE: llvm@22 '- thirdparty/build-thirdparty.sh:55 '- :100-110 on Darwin: DORIS_CLANG_HOME / CC / CXX := $(brew --prefix llvm@20) llvm@20 missing -> exit 1, "brew install llvm@20" -> third-party libraries: llvm@20 run by: build.sh (library missing from thirdparty/installed), manual runs, build-thirdparty.yml (PR check), doris-thirdparty build-target.yml (publisher) brew install lists, i.e. who installs which LLVM: macOS build guide (apache/doris-website#4180) llvm@22 llvm@20 .github/workflows/be-ut-mac.yml llvm@22 llvm@20 .github/workflows/build-thirdparty.yml llvm@20 apache/doris-thirdparty build-target.yml llvm@20 be/CMakeLists.txt if (COMPILER_CLANG) |- add_compile_options(-Wpedantic ...) <- enables the c2y group '- if (CMAKE_CXX_COMPILER_VERSION >= 19) add_compile_options(-Wno-c2y-extensions) <- must stay after the line above | be/src/runtime/runtime_profile.h:73-85 '- MACRO_CONCAT(SCOPED_TIMER, __COUNTER__) <- the only __COUNTER__ use in be/src, be/test | be/src/storage/index/ann/cmake-protect/CMakeLists.txt:48 '- add_subdirectory(contrib/openblas) '- cmake/prebuild.cmake:1513 execute_process(getarch) <- where the hang surfaced ``` ### Release note None ### Check List (For Author) - Test: Manual test on macOS 26.5.1 (arm64, dyld-1378), plus CI. - `clang -fsanitize=address` hello world exits 0 with llvm@22 and still deadlocks with llvm@20 on the same host. - `sh run-be-ut.sh` with no environment overrides now selects `Clang-22.1.8` and clears the `-- Running getarch` point that previously hung forever: zero FAILED targets, and `doris_be_test` links. - `sh run-be-ut.sh --run --filter=FormatRoundTest.*` starts the ASAN binary and passes 8 tests. - `-Wno-c2y-extensions` gate, tested on a `__COUNTER__` TU under `-Wpedantic -Werror`: clang 16.0.6 rejects the flag (`unknown warning option`), which is why it is gated; clang 20.1.8 accepts it; clang 22.1.8 fails without it and passes with it. - `build-thirdparty.sh` compiler override, checked under `bash -x` with both `custom_env.sh` and `build.sh` exporting the `llvm@22` values: the effective `CC`/`CXX` and the first clang on `PATH` are llvm@20. With `llvm@20` missing, the script exits 1 with the brew hint before building anything. - `be-ut-mac.yml`: the "Build BE" step script passes `bash -n` on bash 3.2 and 5.3, and its `cellars` array holds both `llvm@22` and `llvm@20`. The workflow runs only on pushes to master and on schedule, so this PR cannot run it. - `build-support/check-build-hygiene.sh` passes. - CI: `Build Third Party Libraries (macOS)`, `(macOS-arm64)` and `(Linux)` pass at `a955357a30f` (run 36542107899), which already contains the `build-thirdparty.sh` override. The first revision, which moved these builds to llvm@22, failed both macOS jobs on unixODBC. - Regression test / unit test: N/A (toolchain, flag and workflow change). - Behavior changed: Yes, for macOS builds only. - The BE builds with llvm@22 instead of llvm@20. - The third-party libraries, including the published prebuilt, stay on llvm@20. Building them from source now requires llvm@20 installed next to llvm@22; without it, the build stops with a `brew install llvm@20` hint. - Existing macOS developers need to run `brew install llvm@22`. Without it, `env.sh` falls back to Apple clang in `/usr/bin` for newly configured build directories. - Does this need documentation: Yes, apache/doris-website#4180. --------- Co-authored-by: Claude Code <[email protected]>
teranos
added a commit
to teranos/ground
that referenced
this pull request
Oct 27, 2026
… clang 22.1.8 #179's macOS job hung in libacp's checkPhase from part 1 on. Sampled on the runner (macOS 26.6.2), the sanitized test spins before main: AsanInitInternal → InitializeShadowMemory → get_dyld_hdr → dyld_shared_cache_iterate_text_swift → _Block_copy → malloc → AsanInitFromRtl → StaticSpinMutex::LockSlow. compiler-rt before llvm/llvm-project#182943 walks the shared cache with ASan's own init lock held; 22.1.3 is the first release with it (#188913). nixpkgs 80bdc1e5 had 21.1.8 and 22.1.0-rc3. nixpkgs moves to nixos-unstable 3ec63636, whose llvmPackages_22 is 22.1.8, and libacp's derivation and dev shell build with it; the default clang there is still 21.1.8. The move brings ldc 1.42.0, which refuses under -betterC an array literal in a function run at compile time: an enum Bill is rebuilt as one at each use, and strop_test hands literals to oneof and sequence. Those bills are static immutable now, and the literals fixed-size arrays passed as slices. Verified locally, each CI step in order: nix build -L .#ground 209s, make test-mcp 55s, make wind && make test && make test-ug 7s, make test-tools 6s, every one passing; libacp's stdenv is clang-wrapper-22.1.8. Co-Authored-By: Claude Opus 5.5 <[email protected]>
teranos
added a commit
to teranos/ground
that referenced
this pull request
Nov 1, 2026
… clang 22.1.8 #179's macOS job hung in libacp's checkPhase from part 1 on. Sampled on the runner (macOS 26.6.2), the sanitized test spins before main: AsanInitInternal → InitializeShadowMemory → get_dyld_hdr → dyld_shared_cache_iterate_text_swift → _Block_copy → malloc → AsanInitFromRtl → StaticSpinMutex::LockSlow. compiler-rt before llvm/llvm-project#182943 walks the shared cache with ASan's own init lock held; 22.1.3 is the first release with it (#188913). nixpkgs 80bdc1e5 had 21.1.8 and 22.1.0-rc3. nixpkgs moves to nixos-unstable 3ec63636, whose llvmPackages_22 is 22.1.8, and libacp's derivation and dev shell build with it; the default clang there is still 21.1.8. The move brings ldc 1.42.0, which refuses under -betterC an array literal in a function run at compile time: an enum Bill is rebuilt as one at each use, and strop_test hands literals to oneof and sequence. Those bills are static immutable now, and the literals fixed-size arrays passed as slices. Verified locally, each CI step in order: nix build -L .#ground 209s, make test-mcp 55s, make wind && make test && make test-ug 7s, make test-tools 6s, every one passing; libacp's stdenv is clang-wrapper-22.1.8. Co-Authored-By: Claude Opus 5.5 <[email protected]>
teranos
added a commit
to teranos/ground
that referenced
this pull request
Nov 1, 2026
… clang 22.1.8 #179's macOS job hung in libacp's checkPhase from part 1 on. Sampled on the runner (macOS 26.6.2), the sanitized test spins before main: AsanInitInternal → InitializeShadowMemory → get_dyld_hdr → dyld_shared_cache_iterate_text_swift → _Block_copy → malloc → AsanInitFromRtl → StaticSpinMutex::LockSlow. compiler-rt before llvm/llvm-project#182943 walks the shared cache with ASan's own init lock held; 22.1.3 is the first release with it (#188913). nixpkgs 80bdc1e5 had 21.1.8 and 22.1.0-rc3. nixpkgs moves to nixos-unstable 3ec63636, whose llvmPackages_22 is 22.1.8, and libacp's derivation and dev shell build with it; the default clang there is still 21.1.8. The move brings ldc 1.42.0, which refuses under -betterC an array literal in a function run at compile time: an enum Bill is rebuilt as one at each use, and strop_test hands literals to oneof and sequence. Those bills are static immutable now, and the literals fixed-size arrays passed as slices. Verified locally, each CI step in order: nix build -L .#ground 209s, make test-mcp 55s, make wind && make test && make test-ug 7s, make test-tools 6s, every one passing; libacp's stdenv is clang-wrapper-22.1.8. Co-Authored-By: Claude Opus 5.5 <[email protected]>
teranos
added a commit
to teranos/ground
that referenced
this pull request
Nov 1, 2026
… clang 22.1.8 #179's macOS job hung in libacp's checkPhase from part 1 on. Sampled on the runner (macOS 26.6.2), the sanitized test spins before main: AsanInitInternal → InitializeShadowMemory → get_dyld_hdr → dyld_shared_cache_iterate_text_swift → _Block_copy → malloc → AsanInitFromRtl → StaticSpinMutex::LockSlow. compiler-rt before llvm/llvm-project#182943 walks the shared cache with ASan's own init lock held; 22.1.3 is the first release with it (#188913). nixpkgs 80bdc1e5 had 21.1.8 and 22.1.0-rc3. nixpkgs moves to nixos-unstable 3ec63636, whose llvmPackages_22 is 22.1.8, and libacp's derivation and dev shell build with it; the default clang there is still 21.1.8. The move brings ldc 1.42.0, which refuses under -betterC an array literal in a function run at compile time: an enum Bill is rebuilt as one at each use, and strop_test hands literals to oneof and sequence. Those bills are static immutable now, and the literals fixed-size arrays passed as slices. Verified locally, each CI step in order: nix build -L .#ground 209s, make test-mcp 55s, make wind && make test && make test-ug 7s, make test-tools 6s, every one passing; libacp's stdenv is clang-wrapper-22.1.8. Co-Authored-By: Claude Opus 5.5 <[email protected]>
teranos
added a commit
to teranos/ground
that referenced
this pull request
Nov 2, 2026
libacp, which lands next, builds and tests under ASan with clang 22. compiler-rt before llvm/llvm-project#182943 deadlocks ASan at start on macOS 26.4 and later; 22.1.3 is the first release with the fix, and nixpkgs 80bdc1e5 had 22.1.0-rc3. nixpkgs moves to nixos-unstable 3ec63636. The move brings ldc 1.42.0, which refuses under -betterC an array literal in a function run at compile time: an enum Bill is rebuilt as one at each use, and strop_test hands literals to oneof and sequence. Those bills are static immutable now, and the literals fixed-size arrays passed as slices. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_015AEqWghzasmHruCQan1TsL
juanlucasrey
pushed a commit
to CppReflectionAD/clang-p4395
that referenced
this pull request
Nov 5, 2026
Cherry-picked from llvm/llvm-project 2e7d07a. Without it the sanitizers hang at startup on macOS 26. The build in CppReflectionAD/cpp_reflection_ad applied it as tools/toolchain/patches/compiler-rt-dyld-header.patch, leaving this submodule modified after every macOS build. (cherry picked from commit 2e7d07a) Signed-off-by: juanlucasrey <[email protected]>
juanlucasrey
added a commit
to CppReflectionAD/clang-p4395
that referenced
this pull request
Nov 5, 2026
[sanitizer_common] [Darwin] Adopt _dyld_get_dyld_header (llvm#182943)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
There is an issue on recent macOS versions with
GetDyldImageHeaderViaSharedCache, which is fixed by adopting_dyld_get_dyld_header. We weakly declare this to ensure runtimes compile with older SDK (currently on CI bots).rdar://167854578