[DF] Use `ROOT::Internal::TreeUtils` functions in `TreeHeadNode` class by vepadulano · Pull Request #8605 · root-project/root

vepadulano · 2021-07-05T11:53:59Z

This PR addresses third step of #8391

Better support friends with the recently introduced ROOT::Internal::TreeUtils functions

…tions The `get_treename` and `get_inputfiles` methods of `TreeHeadNode` now use the functions from the `ROOT::Internal::TreeUtils` namespace to retrieve information on the input tree of the distributed RDataFrame. If the user did not supply a TTree or TChain object, the functions still need to resort to some parsing of the constructor arguments.

phsft-bot · 2021-07-05T11:54:06Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-07-05T12:31:40Z

Build failed on ROOT-ubuntu16/nortcxxmod.
Running on sft-ubuntu-1604-4.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Failing tests:

projectroot.hist.histdrawv7.test.gtest_hist_histdrawv7_test_histhistdrawv7testUnit

eguiraud

LGTM, only minor comments

bindings/experimental/distrdf/python/DistRDF/Backends/Base.py

bindings/experimental/distrdf/test/test_friendinfo.py

bindings/experimental/distrdf/python/DistRDF/HeadNode.py

bindings/experimental/distrdf/python/DistRDF/Backends/Base.py

This commit removes the `DistRDF.HeadNode.FriendInfo` class in favor of using the information retrieved by the function `ROOT::Internal::TreeUtils::GetFriendInfo`. This function returns a struct of three vectors: 1. a vector of (name, alias) pairs of the friend trees 2. a vector of vectors containing the filenames of each friend tree 3. a vector of vectors containing the tree names of each subtree of a friend TChain, or just an empty vector if the friend is a TTree The logic in the `mapper` function has been also updated to iterate over the information contained in this struct and recreate the friend trees on the workers in a more consistent way then previously done, thus fixing root-project#7584

phsft-bot · 2021-07-05T16:57:51Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-07-05T16:58:10Z

Build failed on ROOT-performance-centos8-multicore/default.
Running on olbdw-01.cern.ch:/data/sftnight/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-05T16:58:09.251Z] CMake Error at /data/sftnight/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1045 (message):

phsft-bot · 2021-07-05T16:58:12Z

Build failed on mac11.0/cxx17.
Running on macphsft20.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-05T16:58:11.835Z] CMake Error at /Users/sftnight/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1045 (message):

phsft-bot · 2021-07-05T16:59:12Z

Build failed on mac1014/python3.
Running on macitois21.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-05T16:59:12.131Z] CMake Error at /Volumes/HD2/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1045 (message):

phsft-bot · 2021-07-05T17:07:47Z

Build failed on ROOT-ubuntu16/nortcxxmod.
Running on sft-ubuntu-1604-4.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-05T17:07:45.756Z] CMake Error at /mnt/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1045 (message):

vepadulano · 2021-07-05T18:51:32Z

@phsft-bot build

phsft-bot · 2021-07-05T18:51:37Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-07-05T18:51:51Z

Build failed on ROOT-performance-centos8-multicore/default.
Running on olbdw-01.cern.ch:/data/sftnight/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-05T18:51:51.053Z] CMake Error at /data/sftnight/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1045 (message):

phsft-bot · 2021-07-05T18:53:26Z

Build failed on mac1014/python3.
Running on macitois21.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-05T18:53:25.005Z] CMake Error at /Volumes/HD2/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1045 (message):

phsft-bot · 2021-07-05T18:54:41Z

Build failed on mac11.0/cxx17.
Running on macphsft20.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-05T18:54:40.301Z] CMake Error at /Users/sftnight/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1045 (message):

phsft-bot · 2021-07-05T18:56:20Z

Build failed on ROOT-ubuntu16/nortcxxmod.
Running on sft-ubuntu-1604-2.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-05T18:56:19.314Z] CMake Error at /mnt/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1045 (message):

phsft-bot · 2021-07-05T20:21:12Z

Build failed on windows10/cxx14.
Running on null:C:\build\workspace\root-pullrequests-build
See console output.

Errors:

[2021-07-05T20:21:11.963Z] CMake Error at C:/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1045 (message):

phsft-bot · 2021-07-05T20:30:41Z

Build failed on ROOT-debian10-i386/cxx14.
Running on pcepsft11.dyndns.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-05T20:30:40.296Z] CMake Error at /home/sftnight/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1045 (message):

vepadulano · 2021-07-06T11:34:20Z

@phsft-bot build

phsft-bot · 2021-07-06T11:34:25Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-07-06T12:23:48Z

Build failed on ROOT-ubuntu16/nortcxxmod.
Running on sft-ubuntu-1604-4.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-06T12:23:47.291Z] CMake Error at /mnt/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1045 (message):

phsft-bot · 2021-07-06T14:09:28Z

Build failed on ROOT-debian10-i386/cxx14.
Running on pcepsft10.dyndns.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-06T14:09:28.092Z] CMake Error at /home/sftnight/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1081 (message):

phsft-bot · 2021-07-06T14:32:44Z

Build failed on ROOT-performance-centos8-multicore/default.
Running on olbdw-01.cern.ch:/data/sftnight/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-06T14:32:43.403Z] CMake Error at /data/sftnight/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1081 (message):

vepadulano · 2021-07-06T15:48:27Z

@phsft-bot build

phsft-bot · 2021-07-06T15:48:33Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-07-06T15:51:21Z

Build failed on ROOT-debian10-i386/cxx14.
Running on pcepsft10.dyndns.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-06T15:51:20.772Z] CMake Error at /home/sftnight/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1081 (message):

phsft-bot · 2021-07-06T15:51:32Z

Build failed on ROOT-ubuntu16/nortcxxmod.
Running on sft-ubuntu-1604-2.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-06T15:51:30.710Z] CMake Error at /mnt/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1081 (message):

phsft-bot · 2021-07-06T16:16:02Z

Build failed on ROOT-performance-centos8-multicore/default.
Running on olbdw-01.cern.ch:/data/sftnight/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-06T16:16:02.038Z] CMake Error at /data/sftnight/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1081 (message):

phsft-bot · 2021-07-06T17:09:49Z

Build failed on mac1014/python3.
Running on macitois21.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-06T17:09:48.963Z] CMake Error at /Volumes/HD2/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1081 (message):

phsft-bot · 2021-07-06T19:09:38Z

Build failed on windows10/cxx14.
Running on null:C:\build\workspace\root-pullrequests-build
See console output.

Errors:

[2021-07-06T19:09:37.670Z] CMake Error at C:/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1045 (message):

phsft-bot · 2021-07-06T19:10:18Z

Build failed on mac11.0/cxx17.
Running on macphsft20.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-06T19:10:17.243Z] CMake Error at /Users/sftnight/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1081 (message):

vepadulano · 2021-07-06T22:22:59Z

@phsft-bot build

phsft-bot · 2021-07-06T22:23:05Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-07-06T22:23:19Z

Build failed on ROOT-performance-centos8-multicore/default.
Running on olbdw-01.cern.ch:/data/sftnight/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-06T22:23:18.345Z] CMake Error at /data/sftnight/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1081 (message):

phsft-bot · 2021-07-06T22:24:15Z

Build failed on ROOT-debian10-i386/cxx14.
Running on pcepsft11.dyndns.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-06T22:24:14.637Z] CMake Error at /home/sftnight/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1081 (message):

phsft-bot · 2021-07-06T22:24:52Z

Build failed on ROOT-ubuntu16/nortcxxmod.
Running on sft-ubuntu-1604-2.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-06T22:24:51.608Z] CMake Error at /mnt/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1081 (message):

phsft-bot · 2021-07-06T22:54:03Z

Build failed on mac11.0/cxx17.
Running on macphsft20.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-06T22:54:02.637Z] CMake Error at /Users/sftnight/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1081 (message):

vepadulano · 2021-07-07T07:41:13Z

Ok so at least one build was successful https://lcgapp-services.cern.ch/root-jenkins/job/root-pullrequests-build/122001/ so I will merge this

vepadulano added the in:RDataFrame label Jul 5, 2021

vepadulano requested review from eguiraud and etejedor July 5, 2021 11:53

vepadulano self-assigned this Jul 5, 2021

vepadulano mentioned this pull request Jul 5, 2021

[DF] Test against reproducer of #7584 root-project/roottest#731

Merged

eguiraud approved these changes Jul 5, 2021

View reviewed changes

bindings/experimental/distrdf/python/DistRDF/Backends/Base.py Show resolved Hide resolved

bindings/experimental/distrdf/test/test_friendinfo.py Outdated Show resolved Hide resolved

etejedor reviewed Jul 5, 2021

View reviewed changes

bindings/experimental/distrdf/python/DistRDF/HeadNode.py Show resolved Hide resolved

etejedor reviewed Jul 5, 2021

View reviewed changes

bindings/experimental/distrdf/python/DistRDF/Backends/Base.py Show resolved Hide resolved

etejedor approved these changes Jul 5, 2021

View reviewed changes

vepadulano added 3 commits July 5, 2021 18:57

[DF] Adapt tests to changes

f3feaee

[DF] Improve tests with friend trees

8e871da

vepadulano force-pushed the distrdf-treeutils-and-improved-friend-support branch from 85a7870 to 8e871da Compare July 5, 2021 16:57

vepadulano merged commit aa21d63 into root-project:master Jul 7, 2021

vepadulano mentioned this pull request Aug 13, 2021

[WIP][DF] Refactor DistRDF range creation #8391

Closed

vepadulano deleted the distrdf-treeutils-and-improved-friend-support branch October 30, 2021 20:56

Conversation

vepadulano commented Jul 5, 2021

Uh oh!

phsft-bot commented Jul 5, 2021

Uh oh!

phsft-bot commented Jul 5, 2021

Failing tests:

Uh oh!

eguiraud left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

phsft-bot commented Jul 5, 2021

Uh oh!

phsft-bot commented Jul 5, 2021

Errors:

Uh oh!

phsft-bot commented Jul 5, 2021

Errors:

Uh oh!

phsft-bot commented Jul 5, 2021

Errors:

Uh oh!

phsft-bot commented Jul 5, 2021

Errors:

Uh oh!

vepadulano commented Jul 5, 2021

Uh oh!

phsft-bot commented Jul 5, 2021

Uh oh!

phsft-bot commented Jul 5, 2021

Errors:

Uh oh!

phsft-bot commented Jul 5, 2021

Errors:

Uh oh!

phsft-bot commented Jul 5, 2021

Errors:

Uh oh!

phsft-bot commented Jul 5, 2021

Errors:

Uh oh!

phsft-bot commented Jul 5, 2021

Errors:

Uh oh!

phsft-bot commented Jul 5, 2021

Errors:

Uh oh!

vepadulano commented Jul 6, 2021

Uh oh!

phsft-bot commented Jul 6, 2021

Uh oh!

phsft-bot commented Jul 6, 2021

Errors:

Uh oh!

phsft-bot commented Jul 6, 2021

Errors:

Uh oh!

phsft-bot commented Jul 6, 2021

Errors:

Uh oh!

vepadulano commented Jul 6, 2021

Uh oh!

phsft-bot commented Jul 6, 2021

Uh oh!

phsft-bot commented Jul 6, 2021

Errors:

Uh oh!

phsft-bot commented Jul 6, 2021

Errors:

Uh oh!

phsft-bot commented Jul 6, 2021

Errors:

Uh oh!

phsft-bot commented Jul 6, 2021

Errors:

Uh oh!