Repository navigation
[Bug] kuromoji_build_dict fails to link on arm64: B/BL out of range (__TEXT exceeds 128MB) #67448
Description
Search before asking
- I had searched in the issues and found no similar issues.
Version
master (ddbaaab1388), macOS 26.5 arm64, Homebrew clang 20.1.8, DISABLE_BUILD_UI=ON sh build.sh --fe --be
What's Wrong?
Linking bin/kuromoji_build_dict fails on arm64 because the binary's __TEXT segment exceeds the AArch64 ±128 MB branch range:
FAILED: [code=1] bin/kuromoji_build_dict
ld: fixup error (kind=arm64_b26) at '__ZN8tcmallocL14memalign_pagesEmmbb'+0x1DC
from libtcmalloc.a[2](libtcmalloc_la-tcmalloc.o), B/BL out of range
(displacement=-135691464, max is +/-128MB), from 0x10816EF18 to 0x100007250 ('___clang_call_terminate')
final section layout:
__TEXT addr=0x100000000, size=0x008310000 <-- 137 MB
__text ... size=0x0059254bc
google_malloc addr=0x10816da40, ...
malloc_hook addr=0x10816ef3c, ...
The linker normally inserts branch islands, but it can only do that inside __text. gperftools puts its allocator in the custom sections google_malloc and malloc_hook, which the linker places after __text, so a branch from there back to ___clang_call_terminate at the start of __TEXT cannot be relaxed.
doris_be itself links fine — ninja doris_be succeeds. Only kuromoji_build_dict is over the limit.
The failure also blocks the install step, which hard-fails when the generated dictionary is absent:
CMake Error at cmake_install.cmake:96 (message):
kuromoji dictionary file .../be/dict/kuromoji/system.bin is missing; build the
'kuromoji_dict' target (stage mecab-ipadic via thirdparty) before packaging.
so output/ is never produced.
What You Expected?
sh build.sh --be completes on arm64.
How to Reproduce?
On an Apple Silicon host:
DISABLE_BUILD_UI=ON sh build.sh --fe --beAnything Else?
kuromoji_build_dict is defined under if (NOT MAKE_TEST) in be/CMakeLists.txt and kuromoji_dict is an ALL target, so a normal --be build links it. be-ut-mac.yml runs ./build.sh --be on runs-on: macos-15 (arm64) with -DMAKE_TEST=OFF, so that CI job should build this target too — worth checking whether it is currently green, since my margin was only ~1.4 MB over the limit and could easily differ with another linker version or flag set.
Ideas, roughly in order of appeal:
- the tool only needs the dictionary builder, not the whole BE; trimming its
target_link_libraries(${DORIS_LINK_LIBS})down to whatkuromoji_build_dict.cppactually uses would take it far under the limit; - link the tool against jemalloc (or no custom allocator) so gperftools' custom sections are not in play;
- ship the generated dictionary as a thirdparty artifact instead of building it during the BE build.
Found while building BE for #67366; worked around locally by dropping ALL from kuromoji_dict and skipping the install-time check.
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Activity
Doris-Breakwater commentedon Sep 2, 2026
Breakwater-GitHub-Analysis-Slot: slot_f68ed80a2dc5
Initial assessment
Triage: confirmed master build regression / macOS arm64 build blocker. The issue is open with no labels or assignee. Suggested existing labels are
kind/fixandarea/aarch64, with routing to the BE build owner.Verified facts
kuromoji_build_dictto the complete${DORIS_LINK_LIBS}closure, andkuromoji_dictis anALLtarget in normal (MAKE_TEST=OFF) builds (CMake target definition). The install failure is therefore downstream: the link failure prevents generation of the four required dictionary files, and the install guard then correctly rejects an incomplete package.build.shdefaultsUSE_JEMALLOC=OFF(selection logic), so a Release build selects and links tcmalloc. Doris already has a macOS-arm64 workaround for the samearm64_b26failure, but it is limited toMAKE_TESTorBUILD_BENCHMARK(existing allocator exclusion); this new tool is created only whenNOT MAKE_TEST, so that protection cannot apply.ddbaaab1388workflow run was canceled by concurrency, but the immediately following master run #153446 failed inBuild BEatbin/kuromoji_build_dictwith the same tcmalloc fixup: displacement-135769416,__TEXTsize0x08324000. The later master run #153448 repeated the identical failure and layout. This independently reproduces the report on the repository'smacos-15arm64 CI image.libtcmalloc.a. Its source directly needs the Kuromoji builder/parser plus DorisStatus; it does not conceptually need the BE service/runtime or a production allocator. The target wiring was introduced by #64667.Root-cause judgment
The evidence supports the issue's mechanism: the new offline tool reuses the whole BE link closure in a normal macOS build, while that build selects gperftools. The resulting Mach-O
__TEXTplaces tcmalloc's customgoogle_malloc/malloc_hooksections beyond the arm64 direct-branch range from___clang_call_terminate; the linker reports that exact out-of-range branch. This is not a missing-dictionary root cause, and it is no longer only a reporter-specific/toolchain hypothesis.The same failure family and allocator workaround already exist in #66615, which is a useful project-local reference.
Recommended next step
${DORIS_LINK_LIBS}or a custom allocator. Audit the actual closure rather than assuming thatStorage/Commonalone is small, because those targets use unity objects and can pull additional unresolved dependencies at archive granularity.kuromoji_build_dicton macOS arm64 and provide the required no-op gperftools symbols using the established [fix](build) Fix the BE unit-test and benchmark build on macOS arm64 #66615 pattern. Keepdoris_beallocator behavior unchanged. This removes the failing custom sections but leaves an unnecessarily large build tool, so it is less maintainable than dependency isolation.kuromoji_dictinALLand keep the install-time file validation. Dropping either only hides the failure and can produce a package without data required by the runtime feature.Evidence required on the fixing PR
ninja -v kuromoji_build_dictshowing the reduced link line, nolibtcmalloc.a, and a comfortably sub-128-MB__TEXT(recordsize/otooloutput before and after).ninja kuromoji_dictfollowed by validation thatsystem.bin,matrix.bin,chardef.bin, andunkdict.binare generated and non-empty.DISABLE_BUILD_UI=ON sh build.sh --beon macOS arm64, including installation intooutput/, plus the relevant Kuromoji BE unit tests on a supported test host.No additional reporter logs or reproduction details are required for initial triage. The remaining unknown is patch-specific: the exact minimal support-library closure must be demonstrated by the candidate link command and clean build, not guessed.
[fix](build) Fix two arm64 BE build failures: kuromoji_build_dict lin…