Use RDatasetSpec in distrdf task creation#10939
Merged
vepadulano merged 2 commits intoroot-project:masterfrom Jul 26, 2022
Merged
Use RDatasetSpec in distrdf task creation#10939vepadulano merged 2 commits intoroot-project:masterfrom
vepadulano merged 2 commits intoroot-project:masterfrom
Conversation
|
This pull request introduces 1 alert when merging 06becc0 into 103525b - view on LGTM.com new alerts:
|
06becc0 to
99036ce
Compare
|
This pull request introduces 1 alert when merging 99036ce into dcf49e4 - view on LGTM.com new alerts:
|
vepadulano
commented
Jul 19, 2022
vepadulano
commented
Jul 19, 2022
At the beginning of a distributed task, move from creating a TChain to creating an RDatasetSpec to pass to the RDataFrame constructor. This simplifies the previous logic by avoiding the need for a TEntryList to restrict the reading of the chain entries to those assigned to the task. This commit also fixes root-project#10872 Previously, each task received a copy of the names of all files of the friend dataset. When restricting the processing to a certain range of entries of the main chain, the friend chain was always being read starting from the first file. Now, in case there are friend trees, each task will receive information about the full dataset and will have to open all files in order to retrieve the number of entries in all trees. This in turn allows proper alignment w.r.t. the friend chain.
58b9f38 to
d427515
Compare
|
Starting build on |
|
Build failed on mac1015/cxx17. Failing tests: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #10872