Try to switch to llvm-12/clang-12 by qoega · Pull Request #20495 · ClickHouse/ClickHouse

qoega · 2021-02-15T06:43:38Z

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

Build/Testing/Packaging Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Switch clang builds to 12 version.

Detailed description / Documentation draft:

...

By adding documentation, you'll allow users to try your new feature immediately, not when someone else will have time to document it later. Documentation is necessary for all features that affect user experience in any way. You can add brief documentation draft above, or add documentation right into your patch as Markdown files in docs folder.

If you are doing this for the first time, it's recommended to read the lightweight Contributing to ClickHouse Documentation guide first.

Information about CI checks: https://clickhouse.tech/docs/en/development/continuous-integration/

qoega · 2021-05-13T17:38:13Z

#24030 should fix gcc build.

mergify · 2021-06-02T17:08:17Z

Command update: success

Branch has been successfully updated

qoega · 2021-06-03T14:37:07Z

0x000000001527fe31 in ZSTD_decompressMultiFrame (dctx=0x7fda1cd7e840, dst=<optimized out>, dstCapacity=<optimized out>, src=0x7fdaa9fd8599, srcSize=52,
    dict=0x0, dictSize=0, ddict=0x0) at ../contrib/zstd/lib/decompress/zstd_decompress.c:995
995	../contrib/zstd/lib/decompress/zstd_decompress.c: No such file or directory.
(gdb) bt
#0  0x000000001527fe31 in ZSTD_decompressMultiFrame (dctx=0x7fda1cd7e840, dst=<optimized out>, dstCapacity=<optimized out>, src=0x7fdaa9fd8599, srcSize=52,
    dict=0x0, dictSize=0, ddict=0x0) at ../contrib/zstd/lib/decompress/zstd_decompress.c:995
#1  0x0000000015281211 in ZSTD_decompress_usingDDict (dctx=0x7fda1cd7e840, dst=0x7fdaf1c39e40, dstCapacity=43, src=0x7fdaa9fd8599, srcSize=52, ddict=0x0)
    at ../contrib/zstd/lib/decompress/zstd_decompress.c:1483
#2  ZSTD_decompressDCtx (dctx=0x7fda1cd7e840, dst=0x7fdaf1c39e40, dstCapacity=43, src=0x7fdaa9fd8599, srcSize=52)
    at ../contrib/zstd/lib/decompress/zstd_decompress.c:1026
#3  ZSTD_decompress (dst=0x7fdaf1c39e40, dstCapacity=43, src=0x7fdaa9fd8599, srcSize=52) at ../contrib/zstd/lib/decompress/zstd_decompress.c:1036
#4  0x000000000f8b727e in DB::CompressionCodecZSTD::doDecompressData (this=<optimized out>, source=0x2b <error: Cannot access memory at address 0x2b>,
    source_size=52, dest=0x7fda1cd7e840 "W\027", uncompressed_size=52) at ../src/Compression/CompressionCodecZSTD.cpp:60
#5  0x000000000f96fc37 in DB::ICompressionCodec::decompress (this=0x7fdaf1c39cd8, source=<optimized out>, source_size=<optimized out>,
    dest=0x7fdaf1c39e40 "\f\025N\217\317\253\061\354I\260o(\205\331\315-\220$") at ../src/Compression/ICompressionCodec.cpp:109
#6  0x000000000f8a199b in DB::CompressedReadBuffer::nextImpl (this=0x7fdae096c1d8) at ../src/Compression/CompressedReadBuffer.cpp:24
#7  0x0000000010122ea8 in DB::ReadBuffer::next (this=0x7fdae096c1d8) at ../src/IO/ReadBuffer.h:62
#8  DB::ReadBuffer::eof (this=0x7fdae096c1d8) at ../src/IO/ReadBuffer.h:93
#9  DB::NativeBlockInputStream::readImpl (this=0x7fda9d630398) at ../src/DataStreams/NativeBlockInputStream.cpp:108
#10 0x000000000f97b5f6 in DB::IBlockInputStream::read (this=0x7fda9d630398) at ../src/DataStreams/IBlockInputStream.cpp:58
#11 0x0000000010b15aba in DB::Connection::receiveDataImpl (this=0x7fda9d64f018, stream=...) at ../src/Client/Connection.cpp:870
#12 0x0000000010b14a87 in DB::Connection::receiveData (this=0x7fda9d64f018) at ../src/Client/Connection.cpp:851
#13 DB::Connection::receivePacket (this=0x7fda9d64f018) at ../src/Client/Connection.cpp:793
#14 0x0000000010b3a2a2 in DB::MultiplexedConnections::receivePacketUnlocked(std::__1::function<void (int, Poco::Timespan, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) (this=0x5, async_callback=...) at ../src/Client/MultiplexedConnections.cpp:285
#15 0x000000000f9a3109 in DB::RemoteQueryExecutorRoutine::operator() (this=0x7fdb4ac46f20, sink=...)
    at ../src/DataStreams/RemoteQueryExecutorReadContext.cpp:46
#16 0x000000000f9a2fce in boost::context::detail::invoke<DB::RemoteQueryExecutorRoutine&, boost::context::fiber> (fn=..., args=...)
    at ../contrib/boost/boost/context/detail/invoke.hpp:41
#17 boost::context::detail::fiber_record<boost::context::fiber, FiberStack&, DB::RemoteQueryExecutorRoutine>::run (this=0x7fdb4ac46f00, fctx=0x5)
    at ../contrib/boost/boost/context/fiber_fcontext.hpp:141
#18 boost::context::detail::fiber_entry<boost::context::detail::fiber_record<boost::context::fiber, FiberStack&, DB::RemoteQueryExecutorRoutine> > (t=...)
    at ../contrib/boost/boost/context/fiber_fcontext.hpp:80
#19 0x0000000008d88e1f in make_fcontext () at ../contrib/boost/libs/context/src/asm/make_x86_64_sysv_elf_gas.S:71
#20 0x0000000000000000 in ?? ()

qoega · 2021-06-03T15:07:42Z

signalHandler (sig=11, info=0x7f11f6d6f4b0, context=0x7f11f6d6f380) at ../base/daemon/BaseDaemon.cpp:129
129	../base/daemon/BaseDaemon.cpp: No such file or directory.
(gdb) bt full
#0  signalHandler (sig=11, info=0x7f11f6d6f4b0, context=0x7f11f6d6f380) at ../base/daemon/BaseDaemon.cpp:129
        buf = '\000' <repeats 1479 times>
        out = <incomplete type>
        signal_context = {uc_flags = 0, uc_link = 0x0, uc_stack = {ss_sp = 0x0, ss_flags = 0, ss_size = 0}, uc_mcontext = {gregs = {0, 0, 0, 2098047, 148481309, 0, 281470681751461, 0 <repeats 12 times>, -3689348814741910323, 16379, 4715951825952350778, 56065}, fpregs = 0x0, __reserved1 = {0, 0, 0, 0, 0, 0, 0, 0}}, uc_sigmask = {__val = {0, 0, 0, 0, 7161137122146610805, 116, 648518346341351424, 0, 538976288, 0, 18446744069431361535, 0, 0, 4985484787499139072, 4985484787499139072, 4985484787500187648}}, __fpregs_mem = {cwd = 0, swd = 16, ftw = 0, fop = 17712, rip = 4318952042648305664, rdp = 4318952042648305664, mxcsr = 17, mxcr_mask = 0, _st = {{significand = {63574, 1, 0, 0}, exponent = 0, __glibc_reserved1 = {0, 0, 49136}}, {significand = {0, 0, 0, 0}, exponent = 0, __glibc_reserved1 = {0, 0, 0}}, {significand = {0, 0, 0, 0}, exponent = 0, __glibc_reserved1 = {0, 0, 0}}, {significand = {0, 0, 0, 0}, exponent = 0, __glibc_reserved1 = {0, 0, 0}}, {significand = {0, 0, 0, 0}, exponent = 22611, __glibc_reserved1 = {18000, 2700, 0}}, {significand = {767, 0, 0, 0}, exponent = 2696, __glibc_reserved1 = {0, 0, 0}}, {significand = {0, 0, 0, 0}, exponent = 0, __glibc_reserved1 = {0, 0, 0}}, {significand = {0, 0, 0, 0}, exponent = 515, __glibc_reserved1 = {0, 0, 0}}}, _xmm = {{element = {0, 0, 0, 0}} <repeats 16 times>}, __glibc_reserved1 = {0 <repeats 24 times>}}}
        stack_trace = {static capacity = 32, size = 0, offset = 0, frame_pointers = {__elems_ = {0x7f127877841b, 0x2b, 0x5, 0x0, 0x7f12c35d6e30, 0x1527fe31 <ZSTD_decompressMultiFrame+113>, 0x10287, 0x2b000000000033, 0x6, 0xe, 0x1000, 0x7f12c35d6e68, 0x7f11f6d6f540, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1000, 0xb, 0x1, 0x7f12c35d6e68, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}}
        saved_errno = <optimized out>
        query_id = <optimized out>
#1  <signal handler called>
No symbol table info available.
#2  0x000000001527fe31 in ZSTD_decompressMultiFrame (dctx=0x7f120e720e40, dst=<optimized out>, dstCapacity=<optimized out>, src=0x7f127877841b, srcSize=52, dict=0x0, dictSize=0, ddict=0x0) at ../contrib/zstd/lib/decompress/zstd_decompress.c:995
        moreThan1Frame = 0
        dststart = <optimized out>
#3  0x0000000015281211 in ZSTD_decompress_usingDDict (dctx=0x7f120e720e40, dst=0x7f1286d743c0, dstCapacity=43, src=0x7f127877841b, srcSize=52, ddict=0x0) at ../contrib/zstd/lib/decompress/zstd_decompress.c:1483
No locals.
#4  ZSTD_decompressDCtx (dctx=0x7f120e720e40, dst=0x7f1286d743c0, dstCapacity=43, src=0x7f127877841b, srcSize=52) at ../contrib/zstd/lib/decompress/zstd_decompress.c:1026
No locals.
#5  ZSTD_decompress (dst=0x7f1286d743c0, dstCapacity=43, src=0x7f127877841b, srcSize=52) at ../contrib/zstd/lib/decompress/zstd_decompress.c:1036
        dctx = 0x7f120e720e40
        regenSize = <optimized out>
#6  0x000000000f8b727e in DB::CompressionCodecZSTD::doDecompressData (this=<optimized out>, source=0x2b <error: Cannot access memory at address 0x2b>, source_size=52, dest=0x7f120e720e40 "", uncompressed_size=52) at ../src/Compression/CompressionCodecZSTD.cpp:60
        res = <optimized out>
#7  0x000000000f96fc37 in DB::ICompressionCodec::decompress (this=0x7f1286d74398, source=<optimized out>, source_size=<optimized out>, dest=0x7f1286d743c0 "\f\025N\217\317\253\061\354I\260o(\205\331\315-\220$") at ../src/Compression/ICompressionCodec.cpp:109
        our_method = <optimized out>
        method = <optimized out>
        decompressed_size = 43
#8  0x000000000f8a199b in DB::CompressedReadBuffer::nextImpl (this=0x7f1286cdb0d8) at ../src/Compression/CompressedReadBuffer.cpp:24
        size_compressed_without_checksum = 0
        size_decompressed = 43
        additional_size_at_the_end_of_buffer = <optimized out>
#9  0x0000000010122ea8 in DB::ReadBuffer::next (this=0x7f1286cdb0d8) at ../src/IO/ReadBuffer.h:62
        res = <optimized out>
#10 DB::ReadBuffer::eof (this=0x7f1286cdb0d8) at ../src/IO/ReadBuffer.h:93
No locals.
#11 DB::NativeBlockInputStream::readImpl (this=0x7f1291a90a18) at ../src/DataStreams/NativeBlockInputStream.cpp:108
        res = <optimized out>
        data_type_factory = <optimized out>
        columns = <optimized out>
        rows = <optimized out>
#12 0x000000000f97b5f6 in DB::IBlockInputStream::read (this=0x7f1291a90a18) at ../src/DataStreams/IBlockInputStream.cpp:58
        res = <optimized out>
#13 0x0000000010b15aba in DB::Connection::receiveDataImpl (this=0x7f1291a33418, stream=...) at ../src/Client/Connection.cpp:870
        external_table_name = {<std::__1::__basic_string_common<true>> = {<No data fields>}, static __short_mask = 128, static __long_mask = 9223372036854775808, __r_ = {<std::__1::__compressed_pair_elem<std::basic_string<char, std::char_traits<char>, std::allocator<char> >::__rep, 0, false>> = {__value_ = {{__l = {__data_ = 0x0, __size_ = 0, __cap_ = 0}, __s = {__data_ = '\000' <repeats 22 times>, {<std::__1::__padding<char, 1>> = {<No data fields>}, __size_ = 0 '\000'}}, __r = {__words = {0, 0, 0}}}}}, <std::__1::__compressed_pair_elem<std::allocator<char>, 1, true>> = {<std::__1::allocator<char>> = {<No data fields>}, <No data fields>}, <No data fields>}, static npos = 18446744073709551615}
        res = <optimized out>
        prev_bytes = <optimized out>
#14 0x0000000010b14a87 in DB::Connection::receiveData (this=0x7f1291a33418) at ../src/Client/Connection.cpp:851
No locals.
#15 DB::Connection::receivePacket (this=0x7f1291a33418) at ../src/Client/Connection.cpp:793
        res = <optimized out>
        e = <optimized out>
#16 0x0000000010b3a2a2 in DB::MultiplexedConnections::receivePacketUnlocked(std::__1::function<void (int, Poco::Timespan, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) (this=0x5, async_callback=...) at ../src/Client/MultiplexedConnections.cpp:285
        async_setter = <optimized out>
        e = <optimized out>
        packet = <optimized out>
        state = @0x7f128563ea00: {connection = 0x7f1291a33418, pool_entry = {data = {__ptr_ = 0x7f128563ead8, __cntrl_ = 0x7f128563eac0}}}
#17 0x000000000f9a3109 in DB::RemoteQueryExecutorRoutine::operator() (this=0x7f12c35f9f20, sink=...) at ../src/DataStreams/RemoteQueryExecutorReadContext.cpp:46
No locals.
#18 0x000000000f9a2fce in boost::context::detail::invoke<DB::RemoteQueryExecutorRoutine&, boost::context::fiber> (fn=..., args=...) at ../contrib/boost/boost/context/detail/invoke.hpp:41
No locals.
#19 boost::context::detail::fiber_record<boost::context::fiber, FiberStack&, DB::RemoteQueryExecutorRoutine>::run (this=0x7f12c35f9f00, fctx=0x5) at ../contrib/boost/boost/context/fiber_fcontext.hpp:141
        c = {fctx_ = 0x0}
#20 boost::context::detail::fiber_entry<boost::context::detail::fiber_record<boost::context::fiber, FiberStack&, DB::RemoteQueryExecutorRoutine> > (t=...) at ../contrib/boost/boost/context/fiber_fcontext.hpp:80
        rec = 0x7f12c35f9f00
        ex = <optimized out>
#21 0x0000000008d88e1f in make_fcontext () at ../contrib/boost/libs/context/src/asm/make_x86_64_sysv_elf_gas.S:71
No locals.
#22 0x0000000000000000 in ?? ()

qoega · 2021-06-03T15:44:39Z

this error is triggered by test 01640_distributed_async_insert_compression

alexey-milovidov · 2021-06-11T23:49:36Z

@Mergifyio update

mergify · 2021-06-11T23:50:48Z

Command update: success

Branch has been successfully updated

nikitamikhaylov · 2021-08-19T18:21:19Z

@Mergifyio update

mergify · 2021-08-19T18:21:29Z

Command update: success

Branch has been successfully updated

qoega · 2021-08-26T15:58:03Z

@Mergifyio update

mergify · 2021-08-26T15:58:26Z

Command update: failure

Base branch update has failed
merge conflict between base and head
err-code: 4BDF5

qoega · 2021-08-26T16:00:40Z

@Mergifyio update

mergify · 2021-08-26T16:00:48Z

Command update: success

Branch already up to date

alesapin · 2021-09-03T11:05:03Z

Looks like problem can be triggered only with -flto=thin. I cannot reproduce with release build without thin-lto. Probably something wrong with linker choice. To be sure I removed clang-11 and lld-11 from image. Let's check what will happen. Next step -- disable thin-lto.

nikitamikhaylov · 2021-09-03T16:12:48Z

@Mergifyio update

mergify · 2021-09-03T16:13:14Z

Command update: success

Branch has been successfully updated

alesapin · 2021-09-04T12:39:41Z

Yes, just incorrect linker was used.

Merging #20495

robot-clickhouse added the pr-build Pull request with build/testing/packaging improvement label Feb 15, 2021

qoega force-pushed the try-llvm-12 branch from ceaa6be to 9cf30b2 Compare February 20, 2021 15:28

Try to switch to llvm-12/clang-12

7e6cd00

qoega force-pushed the try-llvm-12 branch from 140f612 to 7e6cd00 Compare March 24, 2021 14:27

Merge remote-tracking branch 'origin' into try-llvm-12

322a737

alexey-milovidov mentioned this pull request Mar 31, 2021

Enable clang vtables optimization #20151

Merged

qoega added 8 commits April 30, 2021 13:11

Merge remote-tracking branch 'origin' into try-llvm-12

e523162

more

aabf530

find llvm

cf277a6

fasttest change

a4e6a96

Merge remote-tracking branch 'origin' into try-llvm-12

d3b5950

linker path required to be specific one

366a7fe

more

783e9b3

Merge remote-tracking branch 'origin' into try-llvm-12

10b8587

kitaisreal self-assigned this May 20, 2021

qoega and others added 3 commits May 21, 2021 17:42

more

d3149ae

Merge remote-tracking branch 'origin' into try-llvm-12

a2e107c

LLVM remove non internal build

50e2336

robot-ch-test-poll4 added the submodule changed At least one submodule changed in this PR. label May 21, 2021

alexey-milovidov mentioned this pull request May 21, 2021

Switch to clang-tidy-12 (trunk) #18205

Closed

Merge remote-tracking branch 'origin/master' into try-llvm-12

437a66c

robot-ch-test-poll3 removed the submodule changed At least one submodule changed in this PR. label May 24, 2021

qoega added the force tests label May 24, 2021

qoega marked this pull request as ready for review May 24, 2021 15:52

qoega added 5 commits May 25, 2021 12:52

try fix aarch64

07556fa

Merge remote-tracking branch 'origin' into try-llvm-12

0e83723

fix darwin

4f711ee

Merge remote-tracking branch 'origin' into try-llvm-12

594fb7b

try

227eb9f

merge remote-tracking branch 'origin' into try-llvm-12

46023c4

qoega force-pushed the try-llvm-12 branch from ec820b2 to 46023c4 Compare August 6, 2021 16:11

Merge branch 'master' into try-llvm-12

9145dd3

Merge branch 'master' into try-llvm-12

bde834a

nikitamikhaylov mentioned this pull request Aug 23, 2021

[WIP] Improve support for build with clang-13 #28046

Merged

Merge branch 'master' into try-llvm-12

7f7ecb0

alesapin added 2 commits September 3, 2021 14:00

Update Dockerfile

69e7e1d

Update Dockerfile

4fe3909

Merge branch 'master' into try-llvm-12

6ccf905

alesapin mentioned this pull request Sep 4, 2021

Merging #20495 #28616

Merged

alesapin added a commit that referenced this pull request Sep 8, 2021

Merge pull request #28616 from ClickHouse/qoega-try-llvm-12

3b9dae8

Merging #20495

alesapin merged commit 6ccf905 into ClickHouse:master Sep 8, 2021

azat mentioned this pull request Sep 11, 2021

Relax ENABLE_CHECK_HEAVY_BUILDS restrictions on systems under pressure (increase RLIMIT_CPU) #28874

Merged

azat mentioned this pull request Sep 21, 2021

d/control: bump clang to 13 #29213

Merged

Conversation

qoega commented Feb 15, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

qoega commented May 13, 2021

Uh oh!

mergify bot commented Jun 2, 2021

Uh oh!

qoega commented Jun 3, 2021

Uh oh!

qoega commented Jun 3, 2021

Uh oh!

qoega commented Jun 3, 2021

Uh oh!

alexey-milovidov commented Jun 11, 2021

Uh oh!

mergify bot commented Jun 11, 2021

Uh oh!

nikitamikhaylov commented Aug 19, 2021

Uh oh!

mergify bot commented Aug 19, 2021

Uh oh!

qoega commented Aug 26, 2021

Uh oh!

mergify bot commented Aug 26, 2021

Uh oh!

qoega commented Aug 26, 2021

Uh oh!

mergify bot commented Aug 26, 2021

Uh oh!

alesapin commented Sep 3, 2021

Uh oh!

nikitamikhaylov commented Sep 3, 2021

Uh oh!

mergify bot commented Sep 3, 2021

Uh oh!

alesapin commented Sep 4, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

qoega commented Feb 15, 2021 •

edited

Loading