Range types according to headnode types in distributed RDF#8534
Range types according to headnode types in distributed RDF#8534vepadulano merged 3 commits intoroot-project:masterfrom
Conversation
|
Starting build on |
eguiraud
left a comment
eguiraud
left a comment
There was a problem hiding this comment.
LGTM, but see the two comments below
|
Starting build on |
|
This pull request introduces 1 alert when merging edc1d78 into 7c4f932 - view on LGTM.com new alerts:
|
|
The warning from LGTM bot is about the |
|
Starting build on |
2 similar comments
|
Starting build on |
|
Starting build on |
|
Build failed on ROOT-debian10-i386/cxx14. Failing tests: |
The `RangesBuilder` class has been removed in favor of having free functions that serve the same purpose. `get_clustered_ranges` creates ranges for distributed execution based on clusters of some TTree/TChain base dataset. It is called by `TreeHeadNode` `get_balanced_ranges` creates roughly equal-sized chunks of entries. It is called by `EmptySourceHeadNode` The `build_range` methods of the head node classes now call directly the free functions in the `Ranges.py` module. In `TreeHeadNode`, split the work in two steps: first gather the clusters and do some checks against the current value of `npartitions`, then actually compute the ranges.
Various named tuples where used in this file to store information about the dataset. Remove all of them in favor of full Python classes, for documentation purposes.
vepadulano
force-pushed
the
distrdf-separate-ranges
branch
from
7728310 to
303929c
Compare
|
Starting build on |
|
I rebased to clean the history a bit, if tests pass I will merge |
|
Build failed on ROOT-ubuntu16/nortcxxmod. Failing tests: |
|
Build failed on ROOT-debian10-i386/cxx14. Errors:
|
4 participants
This PR addresses step 2 of #8391