Add CI check for PPC64le by abyss7 · Pull Request #25486 · ClickHouse/ClickHouse

abyss7 · 2021-06-18T15:25:39Z

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

Build/Testing/Packaging Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Add CI check for PPC64le cross-compilation

utils/s3tools/s3uploader

mergify · 2021-06-21T16:42:39Z

Command update: success

Branch has been successfully updated

Hey, I reacted but my real name is @Mergifyio

abyss7 · 2021-06-23T14:26:44Z

@Mergifyio update

mergify · 2021-06-23T14:29:20Z

Command update: success

Branch has been successfully updated

alexey-milovidov · 2021-06-23T19:52:36Z

I cannot find it in https://clickhouse-builds.s3.yandex.net/25486/2622fbf9671f669f7560a5546da0ef2c72bdc1b7/clickhouse_special_build_check/report.html

abyss7 · 2021-06-23T20:27:21Z

I cannot find it in https://clickhouse-builds.s3.yandex.net/25486/2622fbf9671f669f7560a5546da0ef2c72bdc1b7/clickhouse_special_build_check/report.html

It's not there yet - I have to add in Arcadia (I guess) after this PR gets merged.

abyss7 · 2021-06-24T12:57:11Z

I cannot find it in https://clickhouse-builds.s3.yandex.net/25486/2622fbf9671f669f7560a5546da0ef2c72bdc1b7/clickhouse_special_build_check/report.html

It's not there yet - I have to add in Arcadia (I guess) after this PR gets merged.

Looks like I'm wrong, I will try to add build via ci_config.json

alexey-milovidov · 2021-06-27T20:33:45Z

Does not work on Ubuntu 20.04 due to too new glibc:

milovidov@milovidov-desktop:~/work/ClickHouse/programs/server$ wget https://clickhouse-builds.s3.yandex.net/25486/2855831adaca71084db89b9a55e896711eba72fa/clickhouse_special_build_check/clang-11-ppc64le_relwithdebuginfo_none_bundled_unsplitted_disable_False_binary/clickhouse
--2021-06-27 23:26:47--  https://clickhouse-builds.s3.yandex.net/25486/2855831adaca71084db89b9a55e896711eba72fa/clickhouse_special_build_check/clang-11-ppc64le_relwithdebuginfo_none_bundled_unsplitted_disable_False_binary/clickhouse
Resolving clickhouse-builds.s3.yandex.net (clickhouse-builds.s3.yandex.net)... 2a02:6b8::2:158, 93.158.134.158
Connecting to clickhouse-builds.s3.yandex.net (clickhouse-builds.s3.yandex.net)|2a02:6b8::2:158|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2039041016 (1,9G) [application/octet-stream]
Saving to: ‘clickhouse’

clickhouse                                           100%[====================================================================================================================>]   1,90G  21,7MB/s    in 89s     

2021-06-27 23:28:16 (21,9 MB/s) - ‘clickhouse’ saved [2039041016/2039041016]

milovidov@milovidov-desktop:~/work/ClickHouse/programs/server$ chmod a+x ./clickhouse
milovidov@milovidov-desktop:~/work/ClickHouse/programs/server$ mv ./clickhouse ./clickhouse-ppc64le
milovidov@milovidov-desktop:~/work/ClickHouse/programs/server$ docker run -it --rm --volume $(pwd):/ClickHouse multiarch/ubuntu-core:ppc64el-wily /ClickHouse/clickhouse-ppc64le
/ClickHouse/clickhouse-ppc64le: /lib/powerpc64le-linux-gnu/libpthread.so.0: version `GLIBC_2.30' not found (required by /ClickHouse/clickhouse-ppc64le)
/ClickHouse/clickhouse-ppc64le: /lib/powerpc64le-linux-gnu/libc.so.6: version `GLIBC_2.27' not found (required by /ClickHouse/clickhouse-ppc64le)
/ClickHouse/clickhouse-ppc64le: /lib/powerpc64le-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /ClickHouse/clickhouse-ppc64le)
/ClickHouse/clickhouse-ppc64le: /lib/powerpc64le-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /ClickHouse/clickhouse-ppc64le)
/ClickHouse/clickhouse-ppc64le: /lib/powerpc64le-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /ClickHouse/clickhouse-ppc64le)
milovidov@milovidov-desktop:~/work/ClickHouse/programs/server$ docker run -it --rm --volume $(pwd):/ClickHouse ppc64le/ubuntu:latest /ClickHouse/clickhouse-ppc64le
Unable to find image 'ppc64le/ubuntu:latest' locally
latest: Pulling from ppc64le/ubuntu
830138a32e2b: Pull complete 
Digest: sha256:b57bc0aa04bd1e6e64199bc146f73db6bc7a7827375a5ca77804dde58d90d887
Status: Downloaded newer image for ppc64le/ubuntu:latest
/ClickHouse/clickhouse-ppc64le: /lib/powerpc64le-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /ClickHouse/clickhouse-ppc64le)
milovidov@milovidov-desktop:~/work/ClickHouse/programs/server$ docker run -it --rm --volume $(pwd):/ClickHouse ppc64le/ubuntu:latest cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.2 LTS"

alexey-milovidov · 2021-06-27T20:40:27Z

docker/packager/binary/Dockerfile

 RUN wget -nv https://clickhouse-datasets.s3.yandex.net/toolchains/toolchains/freebsd-11.3-toolchain.tar.xz

+# Download toolchain for POWER9 (PPC64le)
+RUN wget -nv https://clickhouse-datasets.s3.yandex.net/toolchains/toolchains/advanced-toolchain-14.0-3-ppc64le.tar.xz


Information about the source must be provided.

alexey-milovidov · 2021-06-27T20:44:18Z

$ docker run -it --rm --volume $(pwd):/ClickHouse ppc64le/ubuntu:latest ldd --version
ldd (Ubuntu GLIBC 2.31-0ubuntu9.2) 2.31
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

abyss7 · 2021-06-28T20:53:05Z

I will check why all of those dependencies on glibc version sneaked in - possibly the build-mode is wrong:
ELF 64-bit LSB executable, 64-bit PowerPC or cisco 7500, version 1 (SYSV), dynamically linked

alexey-milovidov · 2021-06-28T22:58:13Z

The toolchain has glibc 2.32 and Ubuntu 20.04 has glibc 2.31 - no surprise.

I recommend instead of using the official toolchain with ~500 MB of assorted files, just extract only some required files from the Ubuntu image (we can extract files directly from the Ubuntu image). Also we can pick some old Ubuntu version like 16.04...

alexey-milovidov

.

CLAassistant · 2021-09-28T10:44:08Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

alexey-milovidov · 2021-10-11T23:11:20Z

Continued in #30010.

abyss7 added 2 commits June 18, 2021 14:44

Add support for ppc64le target

949022a

Update Dockerfile

7de8d06

robot-clickhouse added the pr-build Pull request with build/testing/packaging improvement label Jun 18, 2021

abyss7 commented Jun 18, 2021

View reviewed changes

utils/s3tools/s3uploader Outdated Show resolved Hide resolved

abyss7 and others added 2 commits June 21, 2021 19:41

Update s3uploader

4190890

Merge branch 'master' into power9

87802b4

alexey-milovidov approved these changes Jun 21, 2021

View reviewed changes

alexey-milovidov self-assigned this Jun 21, 2021

Fix PVS warnings

95d189e

abyss7 added the force tests label Jun 23, 2021

Merge branch 'master' into power9

2622fbf

abyss7 and others added 5 commits June 24, 2021 16:06

Add build configuration

be7d0b5

Update ci_config.json

0998fcb

Fix packager args

7432d49

Add SSL support

0fb7227

Merge branch 'master' into power9

2855831

alexey-milovidov reviewed Jun 27, 2021

View reviewed changes

alexey-milovidov requested changes Jul 13, 2021

View reviewed changes

alexey-milovidov mentioned this pull request Aug 21, 2021

Build failure on Power9 arch #25608

Closed

This was referenced Oct 11, 2021

Experiment with minimized toolchain (2) #29974

Merged

Add cross-build for PowerPC 64le #30010

Merged

alexey-milovidov closed this Oct 11, 2021

alexey-milovidov mentioned this pull request Oct 11, 2021

Roadmap 2021 (discussion) #17623

Closed

Conversation

abyss7 commented Jun 18, 2021

Uh oh!

Uh oh!

mergify bot commented Jun 21, 2021

Uh oh!

abyss7 commented Jun 23, 2021

Uh oh!

mergify bot commented Jun 23, 2021

Uh oh!

alexey-milovidov commented Jun 23, 2021

Uh oh!

abyss7 commented Jun 23, 2021

Uh oh!

abyss7 commented Jun 24, 2021

Uh oh!

alexey-milovidov commented Jun 27, 2021

Uh oh!

alexey-milovidov Jun 27, 2021

Choose a reason for hiding this comment

Uh oh!

alexey-milovidov commented Jun 27, 2021

Uh oh!

abyss7 commented Jun 28, 2021

Uh oh!

alexey-milovidov commented Jun 28, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alexey-milovidov left a comment

Choose a reason for hiding this comment

Uh oh!

CLAassistant commented Sep 28, 2021

Uh oh!

alexey-milovidov commented Oct 11, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

alexey-milovidov commented Jun 28, 2021 •

edited

Loading