Skip to content

[builtins] Update builtin xrootd to 5.3.0 and move it to proper ROOT builtins#7335

Merged
linev merged 15 commits intoroot-project:masterfrom
oshadura:builtin-xrootd-5.1.0
Oct 5, 2021
Merged

[builtins] Update builtin xrootd to 5.3.0 and move it to proper ROOT builtins#7335
linev merged 15 commits intoroot-project:masterfrom
oshadura:builtin-xrootd-5.1.0

Conversation

@oshadura
Copy link
Copy Markdown
Collaborator

@oshadura oshadura commented Mar 2, 2021

No description provided.

@phsft-bot
Copy link
Copy Markdown

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-fedora30/cxx14, ROOT-fedora31/noimt, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

@oshadura oshadura self-assigned this Mar 2, 2021
@oshadura oshadura requested a review from Axel-Naumann March 2, 2021 15:09
@phsft-bot
Copy link
Copy Markdown

Build failed on ROOT-ubuntu16/nortcxxmod.
Running on sft-ubuntu-1604-4.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Errors:

  • [2021-03-02T15:09:36.524Z] FAILED: /usr/bin/ccache /usr/bin/c++ -I/mnt/build/workspace/root-pullrequests-build/root/net/netx/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/unix/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/clingutils/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/textinput/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/thread/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/zip/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/rint/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/clib/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/meta/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/gui/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/cont/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/foundation/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/base/inc -Iginclude -I/mnt/build/workspace/root-pullrequests-build/root/net/net/inc -I/mnt/build/workspace/root-pullrequests-build/root/io/io/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/clib/res -I/mnt/build/workspace/root-pullrequests-build/root/builtins -IXROOTD-prefix/include/xrootd -IXROOTD-prefix/include/xrootd/private -I/mnt/build/workspace/root-pullrequests-build/root/proof/proofd/inc -fdiagnostics-color=always -std=c++11 -pipe -Wshadow -Wall -W -Woverloaded-virtual -fsigned-char -pthread -O3 -fPIC -std=c++11 -MD -MT net/netx/CMakeFiles/Netx.dir/src/TXNetFile.cxx.o -MF net/netx/CMakeFiles/Netx.dir/src/TXNetFile.cxx.o.d -o net/netx/CMakeFiles/Netx.dir/src/TXNetFile.cxx.o -c /mnt/build/workspace/root-pullrequests-build/root/net/netx/src/TXNetFile.cxx
  • [2021-03-02T15:09:36.524Z] /mnt/build/workspace/root-pullrequests-build/root/net/netx/src/TXNetFile.cxx:58:34: fatal error: XrdClient/XrdClient.hh: No such file or directory
  • [2021-03-02T15:09:36.524Z] FAILED: /usr/bin/ccache /usr/bin/c++ -I/mnt/build/workspace/root-pullrequests-build/root/net/netx/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/unix/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/clingutils/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/textinput/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/thread/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/zip/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/rint/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/clib/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/meta/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/gui/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/cont/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/foundation/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/base/inc -Iginclude -I/mnt/build/workspace/root-pullrequests-build/root/net/net/inc -I/mnt/build/workspace/root-pullrequests-build/root/io/io/inc -I/mnt/build/workspace/root-pullrequests-build/root/core/clib/res -I/mnt/build/workspace/root-pullrequests-build/root/builtins -IXROOTD-prefix/include/xrootd -IXROOTD-prefix/include/xrootd/private -I/mnt/build/workspace/root-pullrequests-build/root/proof/proofd/inc -fdiagnostics-color=always -std=c++11 -pipe -Wshadow -Wall -W -Woverloaded-virtual -fsigned-char -pthread -O3 -fPIC -std=c++11 -MD -MT net/netx/CMakeFiles/Netx.dir/src/TXNetSystem.cxx.o -MF net/netx/CMakeFiles/Netx.dir/src/TXNetSystem.cxx.o.d -o net/netx/CMakeFiles/Netx.dir/src/TXNetSystem.cxx.o -c /mnt/build/workspace/root-pullrequests-build/root/net/netx/src/TXNetSystem.cxx
  • [2021-03-02T15:09:36.524Z] /mnt/build/workspace/root-pullrequests-build/root/net/netx/src/TXNetSystem.cxx:37:40: fatal error: XrdClient/XrdClientVector.hh: No such file or directory

@phsft-bot
Copy link
Copy Markdown

Build failed on ROOT-fedora31/noimt.
Running on root-fedora-31-2.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build
See console output.

Errors:

  • [2021-03-02T15:25:16.497Z] /home/sftnight/build/workspace/root-pullrequests-build/root/net/netx/src/TXNetSystem.cxx:37:10: fatal error: XrdClient/XrdClientVector.hh: No such file or directory
  • [2021-03-02T15:25:16.497Z] /home/sftnight/build/workspace/root-pullrequests-build/root/net/netx/src/TXNetFile.cxx:58:10: fatal error: XrdClient/XrdClient.hh: No such file or directory

@phsft-bot
Copy link
Copy Markdown

Build failed on mac1014/python3.
Running on macphsft17.dyndns.cern.ch:/build/jenkins/workspace/root-pullrequests-build
See console output.

Errors:

  • [2021-03-02T15:32:17.443Z] FAILED: net/netx/CMakeFiles/Netx.dir/src/TXNetFile.cxx.o
  • [2021-03-02T15:32:17.443Z] /build/jenkins/workspace/root-pullrequests-build/root/net/netx/src/TXNetFile.cxx:58:10: fatal error: 'XrdClient/XrdClient.hh' file not found
  • [2021-03-02T15:32:17.443Z] FAILED: net/netx/CMakeFiles/Netx.dir/src/TXNetSystem.cxx.o
  • [2021-03-02T15:32:17.701Z] /build/jenkins/workspace/root-pullrequests-build/root/net/netx/src/TXNetSystem.cxx:37:10: fatal error: 'XrdClient/XrdClientVector.hh' file not found

@phsft-bot
Copy link
Copy Markdown

Build failed on ROOT-performance-centos8-multicore/default.
Running on null:/data/sftnight/workspace/root-pullrequests-build
See console output.

Errors:

  • [2021-03-02T15:45:04.108Z] /data/sftnight/workspace/root-pullrequests-build/root/net/netx/src/TXNetFile.cxx:58:10: fatal error: XrdClient/XrdClient.hh: No such file or directory
  • [2021-03-02T15:45:04.108Z] /data/sftnight/workspace/root-pullrequests-build/root/net/netx/src/TXNetSystem.cxx:37:10: fatal error: XrdClient/XrdClientVector.hh: No such file or directory

@phsft-bot
Copy link
Copy Markdown

Build failed on ROOT-fedora30/cxx14.
Running on root-fedora30-2.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build
See console output.

Errors:

  • [2021-03-02T16:03:23.700Z] FAILED: net/netx/CMakeFiles/Netx.dir/src/TXNetSystem.cxx.o
  • [2021-03-02T16:03:23.963Z] /home/sftnight/build/workspace/root-pullrequests-build/root/net/netx/src/TXNetSystem.cxx:37:10: fatal error: XrdClient/XrdClientVector.hh: No such file or directory
  • [2021-03-02T16:03:23.963Z] FAILED: net/netx/CMakeFiles/Netx.dir/src/TXNetFile.cxx.o
  • [2021-03-02T16:03:24.284Z] /home/sftnight/build/workspace/root-pullrequests-build/root/net/netx/src/TXNetFile.cxx:58:10: fatal error: XrdClient/XrdClient.hh: No such file or directory

@phsft-bot
Copy link
Copy Markdown

Build failed on mac11.0/cxx17.
Running on macphsft20.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Errors:

  • [2021-03-02T16:15:00.210Z] FAILED: net/netx/CMakeFiles/Netx.dir/src/TXNetFile.cxx.o
  • [2021-03-02T16:15:00.470Z] /Users/sftnight/build/workspace/root-pullrequests-build/root/net/netx/src/TXNetFile.cxx:58:10: fatal error: 'XrdClient/XrdClient.hh' file not found
  • [2021-03-02T16:15:00.470Z] FAILED: net/netx/CMakeFiles/Netx.dir/src/TXNetSystem.cxx.o
  • [2021-03-02T16:15:00.729Z] /Users/sftnight/build/workspace/root-pullrequests-build/root/net/netx/src/TXNetSystem.cxx:37:10: fatal error: 'XrdClient/XrdClientVector.hh' file not found

@phsft-bot
Copy link
Copy Markdown

Build failed on ROOT-debian10-i386/cxx14.
Running on pcepsft10.dyndns.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Errors:

  • [2021-03-02T16:27:39.361Z] /home/sftnight/build/workspace/root-pullrequests-build/root/net/netx/src/TXNetFile.cxx:58:10: fatal error: XrdClient/XrdClient.hh: No such file or directory

@phsft-bot
Copy link
Copy Markdown

Build failed on windows10/cxx14.
Running on null:C:\build\workspace\root-pullrequests-build
See console output.

Failing tests:

@linev
Copy link
Copy Markdown
Member

linev commented Mar 3, 2021

I am facing problem to build xrootd on my platform - latest OpenSUSE Tumblweed.
Maybe latest 4.x XROOTD can solve the problem - I will try

@oshadura
Copy link
Copy Markdown
Collaborator Author

oshadura commented Mar 3, 2021

I am facing problem to build xrootd on my platform - latest OpenSUSE Tumblweed.
Maybe latest 4.x XROOTD can solve the problem - I will try

@linev can you post please what do you see as an error?

@linev
Copy link
Copy Markdown
Member

linev commented Mar 3, 2021

can you post please what do you see as an error?

It is exactly same error as with 4.12.3, which is mentioned in #7292

endif()

set(XROOTD_LIBRARIES ${XROOTD_ROOTDIR}/${XROOTD_LIBDIR}/libXrdUtils${CMAKE_SHARED_LIBRARY_SUFFIX}
${XROOTD_ROOTDIR}/${XROOTD_LIBDIR}/libXrdClient${CMAKE_SHARED_LIBRARY_SUFFIX}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oshadura
libXrdClient no longer provided in the xrootd 5.x,
If I rermove it here, I can compile ROOT on my system.
Version 5.1 is working for me.
But we have to check that net/netx disabled for xrootd 5.x while net/netx based on client library

Copy link
Copy Markdown
Collaborator Author

@oshadura oshadura Mar 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for feedback! I will add your suggestion.

But we have to check that net/netx disabled for xrootd 5.x while net/netx based on client library

it is already done some time ago.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is already done some time ago.

Unfortunately not - if I specify -Dnetx=ON ROOT will try to build it without checking of XROOTD version.
If you want, I can add these fixes to your PR

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got problem when I call cmake second time adding -Dnetx=ON.
But I have anyway problems with incremental builds now, but this is another issue

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And now you move setting of xrootd version number in other cmake file - is it a problem?
I propose, that these checks done directly in net/netx and net/netxng - at the moment when they build

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, let me try to experiment with your suggestion.

@phsft-bot
Copy link
Copy Markdown

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-fedora30/cxx14, ROOT-fedora31/noimt, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

@oshadura
Copy link
Copy Markdown
Collaborator Author

oshadura commented Mar 4, 2021

@Axel-Naumann I would like to add this PR to ROOT 6.24. Would it be still possible?

@phsft-bot
Copy link
Copy Markdown

Build failed on ROOT-performance-centos8-multicore/default.
Running on null:/data/sftnight/workspace/root-pullrequests-build
See console output.

Failing tests:

@phsft-bot
Copy link
Copy Markdown

Build failed on ROOT-fedora30/cxx14.
Running on root-fedora30-2.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build
See console output.

Failing tests:

@phsft-bot
Copy link
Copy Markdown

Build failed on ROOT-ubuntu16/nortcxxmod.
Running on sft-ubuntu-1604-4.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Failing tests:

@phsft-bot
Copy link
Copy Markdown

Build failed on ROOT-fedora31/noimt.
Running on root-fedora-31-2.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build
See console output.

Failing tests:

@phsft-bot
Copy link
Copy Markdown

Build failed on mac1014/python3.
Running on macitois21.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Warnings:

  • [2021-03-04T10:17:38.542Z] XROOTD-prefix/include/xrootd/XrdCl/XrdClOptional.hh:58:29: warning: unused parameter 'n' [-Wunused-parameter]

Failing tests:

@phsft-bot
Copy link
Copy Markdown

Build failed on windows10/cxx14.
Running on null:C:\build\workspace\root-pullrequests-build
See console output.

Failing tests:

@Axel-Naumann
Copy link
Copy Markdown
Member

That depends on when it's in a mergable state. Especially those https://lcgapp-services.cern.ch/root-jenkins/job/root-pullrequests-build/108798/testReport/projectroot.tree.treeplayer/test/gtest_tree_treeplayer_test_treeprocessormt_remotefiles/ and friends seem relevant (IIRC @vepadulano had fixed this in the past?); @eguiraud confirmed that the rdf override / overwrite tests are already fixed.

@vepadulano
Copy link
Copy Markdown
Member

Especially those https://lcgapp-services.cern.ch/root-jenkins/job/root-pullrequests-build/108798/testReport/projectroot.tree.treeplayer/test/gtest_tree_treeplayer_test_treeprocessormt_remotefiles/ and friends seem relevant

These look similar to something that I addressed in the past, that currently is taken care of with https://github.com/root-project/root/blob/master/cmake/modules/XROOTDApplePostInstall.cmake . I tried to look for the messages from that script in the logs of the build (e.g. "Adjusting LC_RPATH variable of XRootD libraries in ${install_libdir}" ) but there was none so maybe that's weird.

That being said, in this particular case the error is that libNetxNG.6.25.01.so doesn't find the reference of @rpath/libXrdUtils.3.dylib to load, whereas in the past the issue was that all libXrD* libraries couldn't load other xrootd libs in the same install folder. So, possibly, it's the RPATH of libNetxNG that should be changed in this case

@linev
Copy link
Copy Markdown
Member

linev commented Mar 4, 2021

#7292 (comment)

Probably 4.12.8 will be better choice for 6.24 release?

@oshadura
Copy link
Copy Markdown
Collaborator Author

oshadura commented Mar 4, 2021

I would prefer xrootd 5 with a lot of new features then xrootd 4.

@phsft-bot
Copy link
Copy Markdown

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

@phsft-bot
Copy link
Copy Markdown

Build failed on mac11.0/cxx17.
Running on macphsft20.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Warnings:

  • [2021-09-24T12:54:05.365Z] builtins/xrootd/XROOTD-prefix/include/xrootd/XrdCl/XrdClOptional.hh:58:29: warning: unused parameter 'n' [-Wunused-parameter]

Failing tests:

@oshadura
Copy link
Copy Markdown
Collaborator Author

@bellenot @Axel-Naumann it think it is ready to be reviewed! Big thanks to @simonmichal!

@phsft-bot
Copy link
Copy Markdown

Build failed on mac1014/python3.
Running on macitois22.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Warnings:

  • [2021-09-24T13:13:54.155Z] builtins/xrootd/XROOTD-prefix/include/xrootd/XrdCl/XrdClOptional.hh:58:29: warning: unused parameter 'n' [-Wunused-parameter]

@Axel-Naumann Axel-Naumann requested a review from linev September 24, 2021 14:54
@Axel-Naumann
Copy link
Copy Markdown
Member

@phsft-bot build with flags -Dbuildin_xrootd=Off

@phsft-bot
Copy link
Copy Markdown

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14 with flags -Dbuildin_xrootd=Off
How to customize builds

@phsft-bot
Copy link
Copy Markdown

Build failed on mac11.0/cxx17.
Running on macphsft20.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Warnings:

  • [2021-09-24T16:17:09.425Z] builtins/xrootd/XROOTD-prefix/include/xrootd/XrdCl/XrdClOptional.hh:58:29: warning: unused parameter 'n' [-Wunused-parameter]

Failing tests:

@phsft-bot
Copy link
Copy Markdown

Build failed on mac1014/python3.
Running on macitois21.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Warnings:

  • [2021-09-24T16:36:14.859Z] builtins/xrootd/XROOTD-prefix/include/xrootd/XrdCl/XrdClOptional.hh:58:29: warning: unused parameter 'n' [-Wunused-parameter]

@Axel-Naumann
Copy link
Copy Markdown
Member

The failures are unrelated.

Copy link
Copy Markdown
Member

@Axel-Naumann Axel-Naumann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks a lot, @oshadura and @simonmichal! We still need @linev 's green light, though - so just commenting to not bypass his review.

@linev
Copy link
Copy Markdown
Member

linev commented Oct 4, 2021

Now ROOT compiles and tests are running.
Last question for @oshadura - was compilation tested with older 4.x xrootd?

@Axel-Naumann
Copy link
Copy Markdown
Member

@phsft-bot build with flags -Dbuiltin_xrootd=Off

@phsft-bot
Copy link
Copy Markdown

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14 with flags -Dbuiltin_xrootd=Off
How to customize builds

@Axel-Naumann
Copy link
Copy Markdown
Member

(@linev : that's what I was trying to find out with disabling the builtin xrootd but I had a spelling mistake argh.)

@phsft-bot
Copy link
Copy Markdown

Build failed on ROOT-debian10-i386/cxx14.
Running on pcepsft10.dyndns.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Errors:

  • [2021-10-04T08:43:22.234Z] CMake Error at cmake/modules/SearchInstalledSoftware.cmake:967 (message):
  • [2021-10-04T08:43:22.234Z] CMake Error at /home/sftnight/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1117 (message):

@phsft-bot
Copy link
Copy Markdown

Build failed on mac1014/python3.
Running on macitois21.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Errors:

  • [2021-10-04T08:44:57.360Z] CMake Error at cmake/modules/SearchInstalledSoftware.cmake:967 (message):
  • [2021-10-04T08:44:57.360Z] CMake Error at /Volumes/HD2/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1117 (message):

@linev
Copy link
Copy Markdown
Member

linev commented Oct 4, 2021

Looks like compilation was ok - on platforms with installed xrood.

@phsft-bot
Copy link
Copy Markdown

Build failed on mac11.0/cxx17.
Running on macphsft23.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Failing tests:

@Axel-Naumann
Copy link
Copy Markdown
Member

Right, ubuntu 16 built with xrootd 4.8.5. @oshadura assigning to you so you have the pleasure to merge :-)

@Axel-Naumann Axel-Naumann assigned oshadura and unassigned jblomer Oct 4, 2021
@Axel-Naumann
Copy link
Copy Markdown
Member

FYI there's still the warning:

builtins/xrootd/XROOTD-prefix/include/xrootd/XrdCl/XrdClOptional.hh:58:29: warning: unused parameter 'n' [-Wunused-parameter]

@simonmichal
Copy link
Copy Markdown

FYI there's still the warning:

builtins/xrootd/XROOTD-prefix/include/xrootd/XrdCl/XrdClOptional.hh:58:29: warning: unused parameter 'n' [-Wunused-parameter]

@Axel-Naumann : I'll fix this in next xrootd release, right now there's not much I can do.

@Axel-Naumann
Copy link
Copy Markdown
Member

Understood but we can (and should) use -isystem for xrootd includes as we can do even less about this! :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

compilation fails on xrootd: _STAT_VER was not declared

10 participants