[cmake] check for internet connection. Fixes #8292 by bellenot · Pull Request #8520 · root-project/root

bellenot · 2021-06-23T11:42:53Z

This should fix an issue with external packages requiring an internet connection when working offline.
If there is no internet connection, they are either automatically disabled or raise a FATAL_ERROR if the fail-on-missing flag is ON

This should fix an issue with external packages requiring an internet connection when working offline. If there is no internet connection, they are either automatically disabled or raise a `FATAL_ERROR` if the `fail-on-missing` flag is `ON`

phsft-bot · 2021-06-23T11:43:00Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-06-23T12:05:56Z

Build failed on ROOT-performance-centos8-multicore/default.
Running on olbdw-01.cern.ch:/data/sftnight/workspace/root-pullrequests-build
See console output.

Failing tests:

pamputt · 2021-06-23T14:04:46Z

cmake/modules/SearchInstalledSoftware.cmake

+      message(FATAL_ERROR "No internet connection. Please check your connection, or either disable the 'builtin_gsl' option or the 'fail-on-missing' to automatically disable options requiring internet access")
+    else()
+      message(STATUS "No internet connection, disabling 'builtin_gsl' option")
+      set(builtin_gsl OFF CACHE BOOL "Disabled because there no internet connection" FORCE)


I think "is" is missing; one should read "Disabled because there is no internet connection"
The same for all the other messages.

Indeed, thanks @pamputt !

phsft-bot · 2021-06-23T14:26:16Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

oshadura

@bellenot super!!! By the way what about other exit codes? I don't remember if there is anything to take in account except of 6? Maybe just to check if exit code is 0 -> TRUE and for others is FALSE?

phsft-bot · 2021-06-23T15:22:08Z

Build failed on ROOT-performance-centos8-multicore/default.
Running on olbdw-01.cern.ch:/data/sftnight/workspace/root-pullrequests-build
See console output.

Failing tests:

bellenot · 2021-06-23T15:24:48Z

@oshadura Well, then I would have to really download a file (that's the only way to get 0 as return code) and then delete it if it is successful. I can do it that way if you prefer (that was my first approach) but I find it a bit too restrictive IMHO...
That was:

#---Try to download a file to check internet connection-----------------------------------------
file(DOWNLOAD https://root.cern/files/test.txt ${CMAKE_CURRENT_BINARY_DIR}/test.txt
  TIMEOUT 100 STATUS DOWNLOAD_STATUS
)
# Get the status code from the download status
list(GET DOWNLOAD_STATUS 0 STATUS_CODE)
# Check if download was successful.
if(${STATUS_CODE} EQUAL 0)
  # success: remove the file and reset the NO_CONNECTION flag
  file(REMOVE ${CMAKE_CURRENT_BINARY_DIR}/test.txt)
  set(NO_CONNECTION FALSE)
else()
  # failure: set the NO_CONNECTION flag
  set(NO_CONNECTION TRUE)
endif()

pcanal · 2021-06-23T15:31:43Z

cmake/modules/SearchInstalledSoftware.cmake


 #---Check for GSL library---------------------------------------------------------------
 if(mathmore OR builtin_gsl)
+  if(builtin_gsl AND NO_CONNECTION)


humm ... we might be missing support for the following scenario:

download and configure (and possibly build once) on a node that has network access

re-configure on the same build directory but from a node that had no network access.

This scenario should be supported as it is a means to download and build on a non-network accessible node.

I.e. we might also need to check whether the buitin has already been download before failing.

Thanks @pcanal for the nice suggestion. i also thought that our CMake system was not complex enough... I'll try to come with something. I already spent several days with the dependencies hell, I can spend several more to try all possible combinations 😉

@pcanal is there documentation somewhere that would explain how to do what you talk about? I may be interested by such option in some cases.
PS: I can start a message on the ROOT forum not to pollute Github with this discussion.

@pcanal I think it's fine to not support this until there is demand, and keep the build system simpler. If people need to build without network we can always ask them to not use builtins. We also have way more important things to work on than a hypothetical corner case of a rare build context...

is there documentation somewhere that would explain how to do what you talk about? I

Not really. It is just: 'on machine that has access to the internet on the disk you need to build on, do a git clone and then a configuration. In some cases that is sufficient and you can continue the build on a machine that has no access to the internet, sometimes you need to do a first (successully) build on the machine with internet access.

phsft-bot · 2021-06-24T11:44:17Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-06-24T12:07:09Z

Build failed on ROOT-performance-centos8-multicore/default.
Running on olbdw-01.cern.ch:/data/sftnight/workspace/root-pullrequests-build
See console output.

Failing tests:

Axel-Naumann · 2021-06-24T12:32:48Z

CMakeLists.txt


+#---Try to download a dummy (non-existing) file to check internet connection--------------------
+file(DOWNLOAD https://root.cern/files/dummy.txt ${CMAKE_CURRENT_BINARY_DIR}/dummy.txt
+  TIMEOUT 100 STATUS DOWNLOAD_STATUS


Is that 100 seconds? I think 10 is enough :-) People might get seriously worried if their CMake just sits there for two minutes. It might also be nice to show

message(STATUS "Checking internet connectivity...") ... if(...) message(STATUS "Yes") else() message(STATUS "No: will not automatically download external dependencies") endif()

@Axel-Naumann :wow: you're right, sorry, I would say that even 3 seconds would be enough... And I will add the messages, thanks for the suggestion

phsft-bot · 2021-06-24T13:08:50Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-06-24T13:39:57Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-06-28T11:10:09Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-06-30T12:05:37Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-06-30T16:45:24Z

Build failed on ROOT-debian10-i386/cxx14.
Running on pcepsft10.dyndns.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Failing tests:

projectroot.roottest.python.cpp.roottest_python_cpp_cpp

phsft-bot · 2021-07-01T12:23:59Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-07-01T13:30:53Z

Build failed on ROOT-debian10-i386/cxx14.
Running on pcepsft11.dyndns.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build
See console output.

Failing tests:

phsft-bot · 2021-07-01T14:31:35Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-07-01T15:51:19Z

Build failed on ROOT-debian10-i386/cxx14.
Running on pcepsft10.dyndns.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Failing tests:

projectroot.roottest.python.cpp.roottest_python_cpp_cpp

phsft-bot · 2021-07-13T16:56:07Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-07-13T17:36:26Z

Build failed on ROOT-debian10-i386/cxx14.
Running on pcepsft10.dyndns.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-13T16:57:45.133Z] stderr: error: could not read '.git/rebase-apply/head-name': No such file or directory

Failing tests:

projectroot.roottest.python.cpp.roottest_python_cpp_cpp

@pamputt

…project#8520) * [cmake] check for internet connection. Fixes root-project#8292 This should fix an issue with external packages requiring an internet connection when working offline. If there is no internet connection, they are either automatically disabled or raise a `FATAL_ERROR` if the `fail-on-missing` flag is `ON` * Add forgotten "is" (thanks @pamputt) * Add internet check for the forgotten 'davix' and 'ssl' options * Change the timeout value and add information messages (thanks Axel) * Let's download an existing file to test internet access * Disable also clad if there is no internet connection * Do not add the clad directory in case of no internet access * Change the logic for clad with no internet connection * disable clad early enough if there is no internet connection * Disable also xrootd when disabling builtin_xrootd

bellenot self-assigned this Jun 23, 2021

bellenot added affects:master in:Build System labels Jun 23, 2021

bellenot requested review from Axel-Naumann and oshadura June 23, 2021 11:43

bellenot mentioned this pull request Jun 23, 2021

Compilation error on redhat 8.3 / no internet #8292

Closed

pamputt reviewed Jun 23, 2021

View reviewed changes

Add forgotten "is" (thanks @pamputt)

bfc0f72

oshadura reviewed Jun 23, 2021

View reviewed changes

pcanal reviewed Jun 23, 2021

View reviewed changes

Add internet check for the forgotten 'davix' and 'ssl' options

aa05264

Axel-Naumann reviewed Jun 24, 2021

View reviewed changes

Axel-Naumann approved these changes Jun 24, 2021

View reviewed changes

Change the timeout value and add information messages (thanks Axel)

62e9f4f

Let's download an existing file to test internet access

c48b9bb

Disable also clad if there is no internet connection

33470bd

Do not add the clad directory in case of no internet access

13860f9

Change the logic for clad with no internet connection

321bbc5

disable clad early enough if there is no internet connection

4255aba

Disable also xrootd when disabling builtin_xrootd

d297824

This was linked to issues Jul 15, 2021

Compilation error on redhat 8.3 / no internet #8292

Closed

Installation without internet access #8424

Closed

bellenot merged commit 49d0be0 into root-project:master Jul 15, 2021

bellenot deleted the check-internet-connection branch February 7, 2022 14:12

Conversation

bellenot commented Jun 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

phsft-bot commented Jun 23, 2021

Uh oh!

phsft-bot commented Jun 23, 2021

Failing tests:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

phsft-bot commented Jun 23, 2021

Uh oh!

oshadura left a comment

Choose a reason for hiding this comment

Uh oh!

phsft-bot commented Jun 23, 2021

Failing tests:

Uh oh!

bellenot commented Jun 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Axel-Naumann Jun 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

phsft-bot commented Jun 24, 2021

Uh oh!

phsft-bot commented Jun 24, 2021

Failing tests:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

phsft-bot commented Jun 24, 2021

Uh oh!

phsft-bot commented Jun 24, 2021

Uh oh!

phsft-bot commented Jun 28, 2021

Uh oh!

phsft-bot commented Jun 30, 2021

Uh oh!

phsft-bot commented Jun 30, 2021

Failing tests:

Uh oh!

phsft-bot commented Jul 1, 2021

Uh oh!

phsft-bot commented Jul 1, 2021

Failing tests:

Uh oh!

phsft-bot commented Jul 1, 2021

Uh oh!

phsft-bot commented Jul 1, 2021

Failing tests:

Uh oh!

phsft-bot commented Jul 13, 2021

Uh oh!

phsft-bot commented Jul 13, 2021

Errors:

Failing tests:

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

bellenot commented Jun 23, 2021 •

edited

Loading

bellenot commented Jun 23, 2021 •

edited

Loading

Axel-Naumann Jun 23, 2021 •

edited

Loading