[WIP] Improve support for build with clang-13 by syominsergey · Pull Request #28046 · ClickHouse/ClickHouse

syominsergey · 2021-08-23T20:28:22Z

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

Build/Testing/Packaging Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Improve support for build with clang-13.

Detailed description / Documentation draft:
This PR continues #27777 and #27714. Some errors with clang-13 are solved here.

syominsergey · 2021-08-23T20:31:06Z

unfortunately, some errors remains, at least this:

semin-serg@semin-serg-ub:~/ClickHouse/build-clang-13$ ninja
[0/2] Re-checking globbed directories...
[15/956] Building CXX object src/CMakeFiles/dbms.dir/Coordination/KeeperStorageDispatcher.cpp.o
FAILED: src/CMakeFiles/dbms.dir/Coordination/KeeperStorageDispatcher.cpp.o 
/usr/bin/clang++-13  -DAWS_SDK_VERSION_MAJOR=1 -DAWS_SDK_VERSION_MINOR=7 -DAWS_SDK_VERSION_PATCH=231 -DBOOST_ASIO_STANDALONE=1 -DCARES_STATICLIB -DCONFIGDIR=\"\" -DENABLE_OPENSSL_ENCRYPTION -DHAVE_CONFIG_H -DLIBSASL_EXPORTS=1 -DLZ4_DISABLE_DEPRECATE_WARNINGS=1 -DOBSOLETE_CRAM_ATTR=1 -DOBSOLETE_DIGEST_ATTR=1 -DPLUGINDIR=\"\" -DPOCO_ENABLE_CPP11 -DPOCO_HAVE_FD_EPOLL -DPOCO_OS_FAMILY_UNIX -DSASLAUTHD_CONF_FILE_DEFAULT=\"\" -DSNAPPY_CODEC_AVAILABLE -DSTD_EXCEPTION_HAS_STACK_TRACE=1 -DUNALIGNED_OK -DUSE_JEMALLOC=1 -DUSE_REPLXX=1 -DUSE_XXHASH=1 -DWITH_COVERAGE=0 -DWITH_GZFILEOP -DX86_64 -DZLIB_COMPAT -D__CLANG_SUPPORT_DYN_ANNOTATION__ -I../contrib/sentry-native/include -Iincludes/configs -I../src -Isrc -Isrc/Core/include -I../base/glibc-compatibility/memcpy -I../base/common/.. -Ibase/common/.. -I../contrib/cityhash102/include -I../contrib/cctz/include -Icontrib/zlib-ng -I../contrib/zlib-ng -I../base/pcg-random/. -I../contrib/aws-c-common/include -I../contrib/aws-c-event-stream/include -Icontrib/aws-s3-cmake/include -Icontrib/cyrus-sasl-cmake -I../contrib/lz4/lib -I../base/mysqlxx/.. -I../contrib/mariadb-connector-c/include -Icontrib/mariadb-connector-c/include -Icontrib/grpc/third_party/cares/cares -I../contrib/grpc/third_party/cares/cares -I../contrib/abseil-cpp -I../contrib/libpq -I../contrib/libpq/include -I../contrib/libstemmer_c/include -I../contrib/wordnet-blast -I../contrib/lemmagen-c/include -isystem ../contrib/libpqxx/include -isystem ../contrib/rocksdb/include -isystem ../contrib/orc/c++/include -isystem contrib/orc/c++/include -isystem ../contrib/AMQP-CPP/include -isystem ../contrib/libuv/include -isystem ../contrib/AMQP-CPP -isystem ../contrib/s2geometry/src/s2 -isystem ../contrib/libhdfs3/include -isystem ../contrib/protobuf/src -isystem ../contrib/sparsehash-c11 -isystem ../contrib/openldap-cmake/linux_x86_64/include -isystem ../contrib/openldap/include -isystem ../contrib/avro/lang/c++/include -isystem ../contrib/miniselect/include -isystem ../contrib/pdqsort -isystem ../contrib/krb5/src/include -isystem contrib/krb5-cmake/include -isystem ../contrib/llvm/llvm/include -isystem contrib/llvm/llvm/include -isystem ../contrib/libcxx/include -isystem ../contrib/libcxxabi/include -isystem ../contrib/libunwind/include -isystem ../contrib/jemalloc-cmake/include -isystem ../contrib/libdivide/. -isystem ../contrib/croaring/cpp -isystem ../contrib/croaring/include -isystem ../contrib/fast_float/include -isystem ../contrib/msgpack-c/include -isystem ../contrib/aws/aws-cpp-sdk-s3/include -isystem ../contrib/aws/aws-cpp-sdk-core/include -isystem ../contrib/xz/src/liblzma/api -isystem ../contrib/zstd/lib -isystem ../contrib/re2 -isystem ../contrib/boost -isystem ../contrib/poco/Net/include -isystem ../contrib/poco/Foundation/include -isystem ../contrib/poco/NetSSL_OpenSSL/include -isystem ../contrib/poco/Crypto/include -isystem ../contrib/boringssl/include -isystem ../contrib/poco/Util/include -isystem ../contrib/poco/JSON/include -isystem ../contrib/poco/XML/include -isystem ../contrib/replxx/include -isystem ../contrib/fmtlib-cmake/../fmtlib/include -isystem ../contrib/double-conversion -isystem ../contrib/dragonbox/include -isystem contrib/re2_st -isystem ../contrib/cppkafka/include -isystem ../contrib/librdkafka-cmake/include -isystem ../contrib/librdkafka/src -isystem contrib/librdkafka-cmake/auxdir -isystem ../contrib/NuRaft/include -isystem ../contrib/poco/MongoDB/include -isystem ../contrib/icu/icu4c/source/i18n -isystem ../contrib/icu/icu4c/source/common -isystem ../contrib/capnproto/c++/src -isystem ../contrib/arrow/cpp/src -isystem ../contrib/arrow-cmake/cpp/src -isystem contrib/arrow-cmake/cpp/src -isystem ../contrib/avro/lang/c++/api -isystem ../contrib/grpc/include -isystem src/Server/grpc_protos -isystem ../contrib/s2geometry/src -isystem ../contrib/sqlite-amalgamation -isystem ../contrib/libc-headers/x86_64-linux-gnu -isystem ../contrib/libc-headers -fdiagnostics-color=always -fsized-deallocation  -gdwarf-aranges -pipe -mssse3 -msse4.1 -msse4.2 -mpclmul -mpopcnt -fasynchronous-unwind-tables -falign-functions=32   -Wall -Wno-unused-command-line-argument  -fdiagnostics-absolute-paths -fexperimental-new-pass-manager -Werror -Wextra -Wframe-larger-than=65536 -Wpedantic -Wno-vla-extension -Wno-zero-length-array -Wno-c11-extensions -Wcomma -Wconditional-uninitialized -Wcovered-switch-default -Wdeprecated -Wembedded-directive -Wempty-init-stmt -Wextra-semi-stmt -Wextra-semi -Wgnu-case-range -Winconsistent-missing-destructor-override -Wnewline-eof -Wold-style-cast -Wrange-loop-analysis -Wredundant-parens -Wreserved-id-macro -Wshadow-field -Wshadow-uncaptured-local -Wshadow -Wstring-plus-int -Wundef -Wunreachable-code-return -Wunreachable-code -Wunused-exception-parameter -Wunused-macros -Wunused-member-function -Wzero-as-null-pointer-constant -Weverything -Wno-c++98-compat-pedantic -Wno-c++98-compat -Wno-c99-extensions -Wno-conversion -Wno-ctad-maybe-unsupported -Wno-deprecated-dynamic-exception-spec -Wno-disabled-macro-expansion -Wno-documentation-unknown-command -Wno-double-promotion -Wno-exit-time-destructors -Wno-float-equal -Wno-global-constructors -Wno-missing-prototypes -Wno-missing-variable-declarations -Wno-nested-anon-types -Wno-packed -Wno-padded -Wno-shift-sign-overflow -Wno-sign-conversion -Wno-switch-enum -Wno-undefined-func-template -Wno-unused-template -Wno-vla -Wno-weak-template-vtables -Wno-weak-vtables -O2 -g -DNDEBUG -O3  -fno-pie   -D OS_LINUX -nostdinc++ -pthread -std=gnu++2a -MD -MT src/CMakeFiles/dbms.dir/Coordination/KeeperStorageDispatcher.cpp.o -MF src/CMakeFiles/dbms.dir/Coordination/KeeperStorageDispatcher.cpp.o.d -o src/CMakeFiles/dbms.dir/Coordination/KeeperStorageDispatcher.cpp.o -c ../src/Coordination/KeeperStorageDispatcher.cpp
In file included from ../src/Coordination/KeeperStorageDispatcher.cpp:1:
In file included from ../src/Coordination/KeeperStorageDispatcher.h:16:
In file included from ../src/Coordination/KeeperServer.h:6:
In file included from ../src/Coordination/KeeperStateMachine.h:3:
In file included from ../src/Coordination/KeeperStorage.h:6:
/home/semin-serg/ClickHouse/src/Common/ZooKeeper/ZooKeeperCommon.h:33:13: error: definition of implicit copy constructor for 'ZooKeeperResponse' is deprecated because it has a user-declared destructor [-Werror,-Wdeprecated-copy-with-dtor]
    virtual ~ZooKeeperResponse() override = default;
            ^
/home/semin-serg/ClickHouse/src/Common/ZooKeeper/ZooKeeperCommon.h:490:8: note: in implicit copy constructor for 'Coordination::ZooKeeperResponse' first required here
struct ZooKeeperSessionIDResponse final : ZooKeeperResponse
       ^
/home/semin-serg/ClickHouse/src/Coordination/KeeperStorageDispatcher.cpp:441:40: note: in implicit copy constructor for 'Coordination::ZooKeeperSessionIDResponse' first required here
            auto session_id_response = dynamic_cast<const Coordination::ZooKeeperSessionIDResponse &>(*response);
                                       ^
1 error generated.
[20/956] Building CXX object src/CMakeFiles/dbms.dir/Processors/QueryPlan/ReadFromMergeTree.cpp.o
ninja: build stopped: subcommand failed.

At the current moment I don't understand what does this means and how to fix this in right way.
I will try to understand and fix this later. Because of this error current PR now in draft status.
I will make it ready to review after fixing all errors.

base/common/LineReader.cpp

base/common/phdr_cache.cpp

base/common/unit.h

base/mysqlxx/CMakeLists.txt

base/readpassphrase/readpassphrase.c

src/Common/ThreadFuzzer.cpp

src/Compression/CompressionCodecDoubleDelta.cpp

src/Compression/CompressionCodecGorilla.cpp

src/IO/ReadBufferFromFileDescriptor.cpp

src/Processors/Formats/Impl/ArrowBufferedStreams.cpp

src/Processors/QueryPlan/ReadFromMergeTree.cpp

nikitamikhaylov · 2021-08-23T23:05:25Z

@syominsergey BTW, you can try to switch CI build to clang-13, if you want. Look at #20495 to see what files are needed to change.

pkit · 2021-08-25T09:07:16Z

unfortunately, some errors remains, at least this:

semin-serg@semin-serg-ub:~/ClickHouse/build-clang-13$ ninja
[0/2] Re-checking globbed directories...
[15/956] Building CXX object src/CMakeFiles/dbms.dir/Coordination/KeeperStorageDispatcher.cpp.o
FAILED: src/CMakeFiles/dbms.dir/Coordination/KeeperStorageDispatcher.cpp.o 
/usr/bin/clang++-13  -DAWS_SDK_VERSION_MAJOR=1 -DAWS_SDK_VERSION_MINOR=7 -DAWS_SDK_VERSION_PATCH=231 -DBOOST_ASIO_STANDALONE=1 -DCARES_STATICLIB -DCONFIGDIR=\"\" -DENABLE_OPENSSL_ENCRYPTION -DHAVE_CONFIG_H -DLIBSASL_EXPORTS=1 -DLZ4_DISABLE_DEPRECATE_WARNINGS=1 -DOBSOLETE_CRAM_ATTR=1 -DOBSOLETE_DIGEST_ATTR=1 -DPLUGINDIR=\"\" -DPOCO_ENABLE_CPP11 -DPOCO_HAVE_FD_EPOLL -DPOCO_OS_FAMILY_UNIX -DSASLAUTHD_CONF_FILE_DEFAULT=\"\" -DSNAPPY_CODEC_AVAILABLE -DSTD_EXCEPTION_HAS_STACK_TRACE=1 -DUNALIGNED_OK -DUSE_JEMALLOC=1 -DUSE_REPLXX=1 -DUSE_XXHASH=1 -DWITH_COVERAGE=0 -DWITH_GZFILEOP -DX86_64 -DZLIB_COMPAT -D__CLANG_SUPPORT_DYN_ANNOTATION__ -I../contrib/sentry-native/include -Iincludes/configs -I../src -Isrc -Isrc/Core/include -I../base/glibc-compatibility/memcpy -I../base/common/.. -Ibase/common/.. -I../contrib/cityhash102/include -I../contrib/cctz/include -Icontrib/zlib-ng -I../contrib/zlib-ng -I../base/pcg-random/. -I../contrib/aws-c-common/include -I../contrib/aws-c-event-stream/include -Icontrib/aws-s3-cmake/include -Icontrib/cyrus-sasl-cmake -I../contrib/lz4/lib -I../base/mysqlxx/.. -I../contrib/mariadb-connector-c/include -Icontrib/mariadb-connector-c/include -Icontrib/grpc/third_party/cares/cares -I../contrib/grpc/third_party/cares/cares -I../contrib/abseil-cpp -I../contrib/libpq -I../contrib/libpq/include -I../contrib/libstemmer_c/include -I../contrib/wordnet-blast -I../contrib/lemmagen-c/include -isystem ../contrib/libpqxx/include -isystem ../contrib/rocksdb/include -isystem ../contrib/orc/c++/include -isystem contrib/orc/c++/include -isystem ../contrib/AMQP-CPP/include -isystem ../contrib/libuv/include -isystem ../contrib/AMQP-CPP -isystem ../contrib/s2geometry/src/s2 -isystem ../contrib/libhdfs3/include -isystem ../contrib/protobuf/src -isystem ../contrib/sparsehash-c11 -isystem ../contrib/openldap-cmake/linux_x86_64/include -isystem ../contrib/openldap/include -isystem ../contrib/avro/lang/c++/include -isystem ../contrib/miniselect/include -isystem ../contrib/pdqsort -isystem ../contrib/krb5/src/include -isystem contrib/krb5-cmake/include -isystem ../contrib/llvm/llvm/include -isystem contrib/llvm/llvm/include -isystem ../contrib/libcxx/include -isystem ../contrib/libcxxabi/include -isystem ../contrib/libunwind/include -isystem ../contrib/jemalloc-cmake/include -isystem ../contrib/libdivide/. -isystem ../contrib/croaring/cpp -isystem ../contrib/croaring/include -isystem ../contrib/fast_float/include -isystem ../contrib/msgpack-c/include -isystem ../contrib/aws/aws-cpp-sdk-s3/include -isystem ../contrib/aws/aws-cpp-sdk-core/include -isystem ../contrib/xz/src/liblzma/api -isystem ../contrib/zstd/lib -isystem ../contrib/re2 -isystem ../contrib/boost -isystem ../contrib/poco/Net/include -isystem ../contrib/poco/Foundation/include -isystem ../contrib/poco/NetSSL_OpenSSL/include -isystem ../contrib/poco/Crypto/include -isystem ../contrib/boringssl/include -isystem ../contrib/poco/Util/include -isystem ../contrib/poco/JSON/include -isystem ../contrib/poco/XML/include -isystem ../contrib/replxx/include -isystem ../contrib/fmtlib-cmake/../fmtlib/include -isystem ../contrib/double-conversion -isystem ../contrib/dragonbox/include -isystem contrib/re2_st -isystem ../contrib/cppkafka/include -isystem ../contrib/librdkafka-cmake/include -isystem ../contrib/librdkafka/src -isystem contrib/librdkafka-cmake/auxdir -isystem ../contrib/NuRaft/include -isystem ../contrib/poco/MongoDB/include -isystem ../contrib/icu/icu4c/source/i18n -isystem ../contrib/icu/icu4c/source/common -isystem ../contrib/capnproto/c++/src -isystem ../contrib/arrow/cpp/src -isystem ../contrib/arrow-cmake/cpp/src -isystem contrib/arrow-cmake/cpp/src -isystem ../contrib/avro/lang/c++/api -isystem ../contrib/grpc/include -isystem src/Server/grpc_protos -isystem ../contrib/s2geometry/src -isystem ../contrib/sqlite-amalgamation -isystem ../contrib/libc-headers/x86_64-linux-gnu -isystem ../contrib/libc-headers -fdiagnostics-color=always -fsized-deallocation  -gdwarf-aranges -pipe -mssse3 -msse4.1 -msse4.2 -mpclmul -mpopcnt -fasynchronous-unwind-tables -falign-functions=32   -Wall -Wno-unused-command-line-argument  -fdiagnostics-absolute-paths -fexperimental-new-pass-manager -Werror -Wextra -Wframe-larger-than=65536 -Wpedantic -Wno-vla-extension -Wno-zero-length-array -Wno-c11-extensions -Wcomma -Wconditional-uninitialized -Wcovered-switch-default -Wdeprecated -Wembedded-directive -Wempty-init-stmt -Wextra-semi-stmt -Wextra-semi -Wgnu-case-range -Winconsistent-missing-destructor-override -Wnewline-eof -Wold-style-cast -Wrange-loop-analysis -Wredundant-parens -Wreserved-id-macro -Wshadow-field -Wshadow-uncaptured-local -Wshadow -Wstring-plus-int -Wundef -Wunreachable-code-return -Wunreachable-code -Wunused-exception-parameter -Wunused-macros -Wunused-member-function -Wzero-as-null-pointer-constant -Weverything -Wno-c++98-compat-pedantic -Wno-c++98-compat -Wno-c99-extensions -Wno-conversion -Wno-ctad-maybe-unsupported -Wno-deprecated-dynamic-exception-spec -Wno-disabled-macro-expansion -Wno-documentation-unknown-command -Wno-double-promotion -Wno-exit-time-destructors -Wno-float-equal -Wno-global-constructors -Wno-missing-prototypes -Wno-missing-variable-declarations -Wno-nested-anon-types -Wno-packed -Wno-padded -Wno-shift-sign-overflow -Wno-sign-conversion -Wno-switch-enum -Wno-undefined-func-template -Wno-unused-template -Wno-vla -Wno-weak-template-vtables -Wno-weak-vtables -O2 -g -DNDEBUG -O3  -fno-pie   -D OS_LINUX -nostdinc++ -pthread -std=gnu++2a -MD -MT src/CMakeFiles/dbms.dir/Coordination/KeeperStorageDispatcher.cpp.o -MF src/CMakeFiles/dbms.dir/Coordination/KeeperStorageDispatcher.cpp.o.d -o src/CMakeFiles/dbms.dir/Coordination/KeeperStorageDispatcher.cpp.o -c ../src/Coordination/KeeperStorageDispatcher.cpp
In file included from ../src/Coordination/KeeperStorageDispatcher.cpp:1:
In file included from ../src/Coordination/KeeperStorageDispatcher.h:16:
In file included from ../src/Coordination/KeeperServer.h:6:
In file included from ../src/Coordination/KeeperStateMachine.h:3:
In file included from ../src/Coordination/KeeperStorage.h:6:
/home/semin-serg/ClickHouse/src/Common/ZooKeeper/ZooKeeperCommon.h:33:13: error: definition of implicit copy constructor for 'ZooKeeperResponse' is deprecated because it has a user-declared destructor [-Werror,-Wdeprecated-copy-with-dtor]
    virtual ~ZooKeeperResponse() override = default;
            ^
/home/semin-serg/ClickHouse/src/Common/ZooKeeper/ZooKeeperCommon.h:490:8: note: in implicit copy constructor for 'Coordination::ZooKeeperResponse' first required here
struct ZooKeeperSessionIDResponse final : ZooKeeperResponse
       ^
/home/semin-serg/ClickHouse/src/Coordination/KeeperStorageDispatcher.cpp:441:40: note: in implicit copy constructor for 'Coordination::ZooKeeperSessionIDResponse' first required here
            auto session_id_response = dynamic_cast<const Coordination::ZooKeeperSessionIDResponse &>(*response);
                                       ^
1 error generated.
[20/956] Building CXX object src/CMakeFiles/dbms.dir/Processors/QueryPlan/ReadFromMergeTree.cpp.o
ninja: build stopped: subcommand failed.

At the current moment I don't understand what does this means and how to fix this in right way.
I will try to understand and fix this later. Because of this error current PR now in draft status.
I will make it ready to review after fixing all errors.

Declaring a "do nothing" destructor prevents auto-creation of a copy constructor in the auto session_id_response = dynamic_cast<const Coordination::ZooKeeperSessionIDResponse &>(*response);
I.e. just remove the destructor declaration or create an explicit copy constructor.

syominsergey · 2021-08-25T20:12:35Z

@pkit thank you for your advice
it seems that original error was solved by means of this commit: 39d99d8

src/Functions/FunctionsCodingIP.cpp

syominsergey · 2021-08-25T20:32:53Z

@nikitamikhaylov, thank you for #28046 (comment)
I can try to switch CI build to clang-13 after solving build problems (I am trying to do this now)

syominsergey · 2021-08-25T22:24:53Z

most of build problems were solved
but some problems remains, at least this:

[225/228] Building CXX object utils/memcpy-bench/CMakeFiles/memcpy-bench.dir/memcpy-bench.cpp.o
		FAILED: utils/memcpy-bench/CMakeFiles/memcpy-bench.dir/memcpy-bench.cpp.o 
		/usr/bin/clang++-13  -DAWS_SDK_VERSION_MAJOR=1 -DAWS_SDK_VERSION_MINOR=7 -DAWS_SDK_VERSION_PATCH=231 -DBOOST_ASIO_STANDALONE=1 -DCARES_STATICLIB -DENABLE_OPENSSL_ENCRYPTION -DPOCO_ENABLE_CPP11 -DPOCO_HAVE_FD_EPOLL -DPOCO_OS_FAMILY_UNIX -DSTD_EXCEPTION_HAS_STACK_TRACE=1 -DUNALIGNED_OK -DUSE_REPLXX=1 -DWITH_COVERAGE=0 -DWITH_GZFILEOP -DX86_64 -DZLIB_COMPAT -D__CLANG_SUPPORT_DYN_ANNOTATION__ -I../contrib/sentry-native/include -Iincludes/configs -I../base/glibc-compatibility/memcpy -I../src -Isrc -Isrc/Core/include -I../base/common/.. -Ibase/common/.. -I../contrib/cityhash102/include -I../contrib/cctz/include -Icontrib/zlib-ng -I../contrib/zlib-ng -I../base/pcg-random/. -I../contrib/aws-c-common/include -I../contrib/aws-c-event-stream/include -Icontrib/aws-s3-cmake/include -I../base/mysqlxx/.. -I../contrib/mariadb-connector-c/include -Icontrib/mariadb-connector-c/include -Icontrib/grpc/third_party/cares/cares -I../contrib/grpc/third_party/cares/cares -I../contrib/abseil-cpp -I../contrib/libpq -I../contrib/libpq/include -I../contrib/libstemmer_c/include -I../contrib/wordnet-blast -I../contrib/lemmagen-c/include -isystem ../contrib/libcxx/include -isystem ../contrib/libcxxabi/include -isystem ../contrib/libunwind/include -isystem ../contrib/libpqxx/include -isystem ../contrib/rocksdb/include -isystem ../contrib/orc/c++/include -isystem contrib/orc/c++/include -isystem ../contrib/AMQP-CPP/include -isystem ../contrib/libuv/include -isystem ../contrib/AMQP-CPP -isystem ../contrib/s2geometry/src/s2 -isystem ../contrib/libhdfs3/include -isystem ../contrib/miniselect/include -isystem ../contrib/pdqsort -isystem ../contrib/llvm/llvm/include -isystem contrib/llvm/llvm/include -isystem ../contrib/croaring/cpp -isystem ../contrib/croaring/include -isystem ../contrib/fast_float/include -isystem ../contrib/msgpack-c/include -isystem ../contrib/aws/aws-cpp-sdk-s3/include -isystem ../contrib/aws/aws-cpp-sdk-core/include -isystem ../contrib/xz/src/liblzma/api -isystem ../contrib/zstd/lib -isystem ../contrib/re2 -isystem ../contrib/boost -isystem ../contrib/poco/Net/include -isystem ../contrib/poco/Foundation/include -isystem ../contrib/poco/NetSSL_OpenSSL/include -isystem ../contrib/poco/Crypto/include -isystem ../contrib/boringssl/include -isystem ../contrib/poco/Util/include -isystem ../contrib/poco/JSON/include -isystem ../contrib/poco/XML/include -isystem ../contrib/replxx/include -isystem ../contrib/fmtlib-cmake/../fmtlib/include -isystem ../contrib/double-conversion -isystem ../contrib/dragonbox/include -isystem contrib/re2_st -isystem ../contrib/NuRaft/include -isystem ../contrib/grpc/include -isystem ../contrib/protobuf/src -isystem src/Server/grpc_protos -isystem ../contrib/s2geometry/src -isystem ../contrib/sqlite-amalgamation -isystem ../contrib/libc-headers/x86_64-linux-gnu -isystem ../contrib/libc-headers -fdiagnostics-color=always -fsized-deallocation  -gdwarf-aranges -pipe -mssse3 -msse4.1 -msse4.2 -mpclmul -mpopcnt -fasynchronous-unwind-tables -falign-functions=32   -Wall -Wno-unused-command-line-argument  -fdiagnostics-absolute-paths -fexperimental-new-pass-manager -Werror -Wextra -Wframe-larger-than=65536 -Wpedantic -Wno-vla-extension -Wno-zero-length-array -Wno-c11-extensions -Wcomma -Wconditional-uninitialized -Wcovered-switch-default -Wdeprecated -Wembedded-directive -Wempty-init-stmt -Wextra-semi-stmt -Wextra-semi -Wgnu-case-range -Winconsistent-missing-destructor-override -Wnewline-eof -Wold-style-cast -Wrange-loop-analysis -Wredundant-parens -Wreserved-id-macro -Wshadow-field -Wshadow-uncaptured-local -Wshadow -Wstring-plus-int -Wundef -Wunreachable-code-return -Wunreachable-code -Wunused-exception-parameter -Wunused-macros -Wunused-member-function -Wzero-as-null-pointer-constant -Weverything -Wno-c++98-compat-pedantic -Wno-c++98-compat -Wno-c99-extensions -Wno-conversion -Wno-ctad-maybe-unsupported -Wno-deprecated-dynamic-exception-spec -Wno-disabled-macro-expansion -Wno-documentation-unknown-command -Wno-double-promotion -Wno-exit-time-destructors -Wno-float-equal -Wno-global-constructors -Wno-missing-prototypes -Wno-missing-variable-declarations -Wno-nested-anon-types -Wno-packed -Wno-padded -Wno-shift-sign-overflow -Wno-sign-conversion -Wno-switch-enum -Wno-undefined-func-template -Wno-unused-template -Wno-vla -Wno-weak-template-vtables -Wno-weak-vtables -O2 -g -DNDEBUG -O3  -fno-pie   -D OS_LINUX -nostdinc++ -pthread -std=gnu++2a -MD -MT utils/memcpy-bench/CMakeFiles/memcpy-bench.dir/memcpy-bench.cpp.o -MF utils/memcpy-bench/CMakeFiles/memcpy-bench.dir/memcpy-bench.cpp.o.d -o utils/memcpy-bench/CMakeFiles/memcpy-bench.dir/memcpy-bench.cpp.o -c ../utils/memcpy-bench/memcpy-bench.cpp
		/home/semin-serg/ClickHouse/utils/memcpy-bench/memcpy-bench.cpp:793:19: error: identifier '__memcpy_erms' is reserved because it starts with '__' [-Werror,-Wreserved-identifier]
		extern "C" void * __memcpy_erms(void * __restrict destination, const void * __restrict source, size_t size);
		                  ^
		/home/semin-serg/ClickHouse/utils/memcpy-bench/memcpy-bench.cpp:794:19: error: identifier '__memcpy_sse2_unaligned' is reserved because it starts with '__' [-Werror,-Wreserved-identifier]
		extern "C" void * __memcpy_sse2_unaligned(void * __restrict destination, const void * __restrict source, size_t size);
		                  ^
		/home/semin-serg/ClickHouse/utils/memcpy-bench/memcpy-bench.cpp:795:19: error: identifier '__memcpy_ssse3' is reserved because it starts with '__' [-Werror,-Wreserved-identifier]
		extern "C" void * __memcpy_ssse3(void * __restrict destination, const void * __restrict source, size_t size);
		                  ^
		/home/semin-serg/ClickHouse/utils/memcpy-bench/memcpy-bench.cpp:796:19: error: identifier '__memcpy_ssse3_back' is reserved because it starts with '__' [-Werror,-Wreserved-identifier]
		extern "C" void * __memcpy_ssse3_back(void * __restrict destination, const void * __restrict source, size_t size);
		                  ^
		/home/semin-serg/ClickHouse/utils/memcpy-bench/memcpy-bench.cpp:797:19: error: identifier '__memcpy_avx_unaligned' is reserved because it starts with '__' [-Werror,-Wreserved-identifier]
		extern "C" void * __memcpy_avx_unaligned(void * __restrict destination, const void * __restrict source, size_t size);
		                  ^
		/home/semin-serg/ClickHouse/utils/memcpy-bench/memcpy-bench.cpp:798:19: error: identifier '__memcpy_avx_unaligned_erms' is reserved because it starts with '__' [-Werror,-Wreserved-identifier]
		extern "C" void * __memcpy_avx_unaligned_erms(void * __restrict destination, const void * __restrict source, size_t size);
		                  ^
		/home/semin-serg/ClickHouse/utils/memcpy-bench/memcpy-bench.cpp:799:19: error: identifier '__memcpy_avx512_unaligned' is reserved because it starts with '__' [-Werror,-Wreserved-identifier]
		extern "C" void * __memcpy_avx512_unaligned(void * __restrict destination, const void * __restrict source, size_t size);
		                  ^
		/home/semin-serg/ClickHouse/utils/memcpy-bench/memcpy-bench.cpp:800:19: error: identifier '__memcpy_avx512_unaligned_erms' is reserved because it starts with '__' [-Werror,-Wreserved-identifier]
		extern "C" void * __memcpy_avx512_unaligned_erms(void * __restrict destination, const void * __restrict source, size_t size);
		                  ^
		/home/semin-serg/ClickHouse/utils/memcpy-bench/memcpy-bench.cpp:801:19: error: identifier '__memcpy_avx512_no_vzeroupper' is reserved because it starts with '__' [-Werror,-Wreserved-identifier]
		extern "C" void * __memcpy_avx512_no_vzeroupper(void * __restrict destination, const void * __restrict source, size_t size);
		                  ^
		9 errors generated.
		[226/228] Building CXX object utils/keeper-bench/CMakeFiles/keeper-bench.dir/Generator.cpp.o
		ninja: build stopped: subcommand failed.

I started to work with __memcpy_erms and renamed it into memcpy_erms in files:

./utils/memcpy-bench/memcpy-bench.cpp
./utils/memcpy-bench/glibc/memmove-vec-unaligned-erms.S

that solved problem with __memcpy_erms

but problem with __memcpy_sse2_unaligned is more tricky
because this is the only match when search __memcpy_sse2_unaligned after renaming __memcpy_sse2_unaligned into memcpy_sse2_unaligned in file ./utils/memcpy-bench/memcpy-bench.cpp:

semin-serg@semin-serg-ub:~/ClickHouse$ grep -r "__memcpy_sse2_unaligned" .
		Binary file ./build-clang-13/utils/memcpy-bench/CMakeFiles/memcpy-bench.dir/glibc/memmove-sse2-unaligned-erms.S.o matches

I don't understand now the source of __memcpy_sse2_unaligned in memmove-sse2-unaligned-erms.S.o because there is no such function in ./utils/memcpy-bench/glibc/memmove-sse2-unaligned-erms.S

I solved this problem with commit fb374fa

syominsergey · 2021-08-25T22:26:15Z

Now local build with clang-13 works in this branch
Will rebase it onto master and check again

syominsergey · 2021-08-26T05:45:57Z

Local build with clang-13 after rebasing onto master was successful.
@nikitamikhaylov what do you think: it is better to setup CI build with clang-13 in this PR or in another one?
my question is related to your comment #28046 (comment)

src/Common/ZooKeeper/ZooKeeperCommon.h

src/Functions/TargetSpecific.h

src/IO/tests/gtest_bit_io.cpp

utils/memcpy-bench/memcpy-bench.cpp

syominsergey · 2021-09-10T08:43:48Z

It looks like undefined symbols when linking contrib/llvm/llvm/bin/llvm-tblgen like _Unwind_Resume, _Unwind_SetGR etc were fixed.
Thank you, @nikitamikhaylov !

syominsergey · 2021-09-10T08:47:05Z

but new strange errors occured
report, log:

Err:6 https://apt.llvm.org/focal llvm-toolchain-focal-13/main amd64 Packages
  File has unexpected size (9881 != 9899). Mirror sync in progress? [IP: 2a04:4e42:3a::561 443]
  Hashes of expected file:
   - Filesize:9899 [weak]
   - SHA256:c3b2ad2ccbd38d8c49b1c83e44be4b1d6cc25e01e11cf49547750468bed18cef
   - SHA1:c4fe26c01c33a1ba4336e4379c9c9bac4c0f0c02 [weak]
   - MD5Sum:8c52b72764b293f076e28b10e84bf353 [weak]
  Release file created at: Thu, 09 Sep 2021 10:01:46 +0000
Fetched 5526 B in 1s (8660 B/s)
Reading package lists...
�[91mE: Failed to fetch https://apt.llvm.org/focal/dists/llvm-toolchain-focal-13/main/binary-amd64/Packages.gz  File has unexpected size (9881 != 9899). Mirror sync in progress? [IP: 2a04:4e42:3a::561 443]
   Hashes of expected file:
    - Filesize:9899 [weak]
    - SHA256:c3b2ad2ccbd38d8c49b1c83e44be4b1d6cc25e01e11cf49547750468bed18cef
    - SHA1:c4fe26c01c33a1ba4336e4379c9c9bac4c0f0c02 [weak]
    - MD5Sum:8c52b72764b293f076e28b10e84bf353 [weak]
   Release file created at: Thu, 09 Sep 2021 10:01:46 +0000

locally on my notebook I have same problems when trying to do apt update
maybe this is really llvm server problems

And tests from testflows failed, see here.
I will look at them more carefully later.

base/daemon/BaseDaemon.cpp

nikitamikhaylov · 2021-09-10T12:05:23Z

@Mergifyio update

mergify · 2021-09-10T12:05:55Z

Command update: success

Branch has been successfully updated

syominsergey · 2021-09-11T10:14:29Z

I investigated problem with test 01492_format_readable_quantity (sql, reference answer).
Problem can be reproduced locally when running test with result as in CI:

semin-serg@semin-serg-ub:~/ClickHouse$ PATH=$PATH:cmake-build-debug-clang-13/programs tests/clickhouse-test 01492_format_readable_quantity
WARNING: jinja2 not installed! Template tests will be skipped.
Using queries from '/home/semin-serg/ClickHouse/tests/queries' directory
Using clickhouse-client as client program (expecting split build)
Connecting to ClickHouse server... OK
Running 1 stateless tests (MainProcess).
01492_format_readable_quantity:                                         [ FAIL ] - result differs with reference:
--- /home/semin-serg/ClickHouse/tests/queries/0_stateless/01492_format_readable_quantity.reference      2021-08-08 23:23:40.014192070 +0300
+++ /home/semin-serg/ClickHouse/tests/queries/0_stateless/01492_format_readable_quantity.stdout 2021-09-11 11:38:23.639276405 +0300
@@ -43,8 +43,8 @@
 1739.27 quadrillion    1739.27 quadrillion     -2.15 billion
 4727.84 quadrillion    4727.84 quadrillion     -2.15 billion
 12851.60 quadrillion   12851.60 quadrillion    -2.15 billion
-34934.27 quadrillion   0.00    -2.15 billion
-94961.19 quadrillion   0.00    -2.15 billion
-258131.29 quadrillion  0.00    -2.15 billion
-701673.59 quadrillion  0.00    -2.15 billion
-1907346.57 quadrillion 0.00    -2.15 billion
+34934.27 quadrillion   9223.37 quadrillion     -2.15 billion
+94961.19 quadrillion   9223.37 quadrillion     -2.15 billion
+258131.29 quadrillion  9223.37 quadrillion     -2.15 billion
+701673.59 quadrillion  9223.37 quadrillion     -2.15 billion
+1907346.57 quadrillion 9223.37 quadrillion     -2.15 billion
Database: test_999xtw
Having 1 errors! 0 tests passed. 0 tests skipped. 0.16 s elapsed (MainProcess).
Won't run stateful tests because test data wasn't loaded.
All tests have finished.

We can visually see the cause of problem using this test query:

select number, exp(number), round(exp(number), 6), toUInt64(round(exp(number), 6)) from system.numbers limit 10 offset 40;

when running on clickhouse build with sources from this PR:

semin-serg@semin-serg-ub:~/ClickHouse$ ./cmake-build-debug-clang-13/programs/clickhouse-client 
ClickHouse client version 21.11.1.1.
Connecting to localhost:9000 as user default.
Connected to ClickHouse server version 21.11.1 revision 54449.
Warnings:
* Server was built in debug mode. It will work slowly.
semin-serg-ub :) select number, exp(number), round(exp(number), 6), toUInt64(round(exp(number), 6)) from system.numbers limit 10 offset 40;
SELECT
    number,
    exp(number),
    round(exp(number), 6),
    toUInt64(round(exp(number), 6))
FROM system.numbers
LIMIT 40, 10
Query id: a39b4e3d-32c8-49ed-be79-90a8eee281c2
┌─number─┬───────────exp(number)─┬─round(exp(number), 6)─┬─toUInt64(round(exp(number), 6))─┐
│     40 │    235385266837188400 │    235385266837188400 │              235385266837188416 │
│     41 │    639843493530239200 │    639843493530239200 │              639843493530239232 │
│     42 │   1739274941519067600 │   1739274941519067600 │             1739274941519067648 │
│     43 │   4727839468226178000 │   4727839468226178000 │             4727839468226178048 │
│     44 │  12851600114360117000 │  12851600114360117000 │            12851600114360117248 │
│     45 │  34934271057471490000 │  34934271057471490000 │             9223372036854775808 │
│     46 │  94961194206095700000 │  94961194206095700000 │             9223372036854775808 │
│     47 │ 258131288619054800000 │ 258131288619054800000 │             9223372036854775808 │
│     48 │ 701673591209169100000 │ 701673591209169100000 │             9223372036854775808 │
│     49 │  1.907346572496092e21 │  1.907346572496092e21 │             9223372036854775808 │
└────────┴───────────────────────┴───────────────────────┴─────────────────────────────────┘
10 rows in set. Elapsed: 0.014 sec.

when running on stable clickhouse from official deb repository:

semin-serg@semin-serg-ub:~/ClickHouse$ clickhouse-client 
ClickHouse client version 21.9.2.17 (official build).
Connecting to localhost:9000 as user default.
Connected to ClickHouse server version 21.9.2 revision 54449.
semin-serg-ub :) select number, exp(number), round(exp(number), 6), toUInt64(round(exp(number), 6)) from system.numbers limit 10 offset 40;
SELECT
    number,
    exp(number),
    round(exp(number), 6),
    toUInt64(round(exp(number), 6))
FROM system.numbers
LIMIT 40, 10
Query id: 4b6fcd28-4440-4717-a920-f84b40524321
┌─number─┬───────────exp(number)─┬─round(exp(number), 6)─┬─toUInt64(round(exp(number), 6))─┐
│     40 │    235385266837188400 │    235385266837188400 │              235385266837188416 │
│     41 │    639843493530239400 │    639843493530239400 │              639843493530239360 │
│     42 │   1739274941519067600 │   1739274941519067600 │             1739274941519067648 │
│     43 │   4727839468226178000 │   4727839468226178000 │             4727839468226178048 │
│     44 │  12851600114360117000 │  12851600114360117000 │            12851600114360117248 │
│     45 │  34934271057471490000 │  34934271057471490000 │                               0 │
│     46 │  94961194206095700000 │  94961194206095700000 │                               0 │
│     47 │ 258131288619054800000 │ 258131288619054800000 │                               0 │
│     48 │ 701673591209169100000 │ 701673591209169100000 │                               0 │
│     49 │  1.907346572496092e21 │  1.907346572496092e21 │                               0 │
└────────┴───────────────────────┴───────────────────────┴─────────────────────────────────┘
10 rows in set. Elapsed: 0.004 sec.

as we can see, the result of function toUInt64 is different for big arguments: in stable clickhouse it returns 0, in clickhouse built from sources from this pr it returns 9223372036854775808.

I have not seen definition of desired behaviour of toUInt64 for such cases in documentation.

So I need help with question - is it really important to keep behaviour of toUInt64 for cases under discussion as in stable clickhouse or this is not important behaviour, so it is better to fix reference answer?

alexey-milovidov · 2021-09-11T12:44:48Z

Transforming out of range Float to Int is unspecified behaviour. Both zero and int max (and any other number) are correct results. The test can be adapted to not depend on this result.

syominsergey · 2021-09-11T15:03:17Z

@Mergifyio update

mergify · 2021-09-11T15:03:53Z

Command update: success

Branch has been successfully updated

syominsergey · 2021-09-11T15:18:55Z

@alexey-milovidov Alexey, thank you for your advice!
I have removed test cases with unspecified behaviour when converting from Float to UInt64 because of out of range here d2c6789
please, let me know if there is some better way to adapt test to not depend on unspecified behaviour

syominsergey · 2021-09-11T21:16:36Z

see some errors in special builds here
will try to fix them

syominsergey · 2021-09-12T16:31:25Z

@Mergifyio update

mergify · 2021-09-12T16:31:58Z

Command update: success

Branch already up to date

robot-clickhouse added the pr-build Pull request with build/testing/packaging improvement label Aug 23, 2021