Skip to content

[ntuple] DAOS object class customization (2/2): provide RNTupleWriteOptionsDaos#8452

Merged
jalopezg-git merged 6 commits intoroot-project:masterfrom
jalopezg-git:ntuple-daos-oclass-impl
Jun 18, 2021
Merged

[ntuple] DAOS object class customization (2/2): provide RNTupleWriteOptionsDaos#8452
jalopezg-git merged 6 commits intoroot-project:masterfrom
jalopezg-git:ntuple-daos-oclass-impl

Conversation

@jalopezg-git
Copy link
Copy Markdown
Contributor

This pull-request is a follow-up of #8402. RNTupleWriteOptionsDaos, i.e. a subclass of RNTupleWriteOptions, may now be used to provide DAOS-specific options. Currently, we only support setting the object class used for user data OIDs.

Changes or fixes:

  • Contains the required changes to support backend-dependent options: classes derived from RNTupleWriteOptions may carry additional options.
  • String serialization/deserialization routines moved to the ROOT::Experimental::Internal::RNTupleSerialization namespace, where they can be reused.
  • RPageStorageDaos: the provided object class, if given, is used for generating the OIDs that relate to user data. OC_SX is always used to store metadata (i.e. header/footer and anchor). The object class must be known to read the ntuple; thus, the object class name is stored in the anchor.
  • Changed the default object class for user data to the more conservative OC_SX (before was OC_RP_XSF). OC_SX distributes data over all the targets and is probably a more sane default.

This PR closes issue #8205.

@phsft-bot
Copy link
Copy Markdown

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

@jalopezg-git jalopezg-git linked an issue Jun 16, 2021 that may be closed by this pull request
Copy link
Copy Markdown
Contributor

@mxxo mxxo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey Javier, just looking at the write options changes everything looks good. It looks like you've been very careful to make potential null dereferences of fOptions impossible. I believe this is the first time dynamic_cast is used in RNTuple, is that is a milestone?

Copy link
Copy Markdown
Contributor

@jblomer jblomer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very well! One tiny comment.

@phsft-bot
Copy link
Copy Markdown

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

@jalopezg-git
Copy link
Copy Markdown
Contributor Author

Thanks for the quick review, @mxxo and @jblomer! Let's see if we can merge this today :-).

@jalopezg-git jalopezg-git requested a review from jblomer June 18, 2021 09:18
@phsft-bot
Copy link
Copy Markdown

Build failed on mac11.0/cxx17.
Running on macphsft20.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Errors:

  • [2021-06-18T09:31:53.784Z] FAILED: tree/ntuple/G__ROOTNTuple.cxx lib/ROOTNTuple.pcm
  • [2021-06-18T09:31:54.429Z] /Users/sftnight/build/workspace/root-pullrequests-build/build/include/ROOT/RNTupleOptions.hxx:90:58: error: 'const' qualifier may not appear after the virtual specifier 'override'

@phsft-bot
Copy link
Copy Markdown

Build failed on ROOT-debian10-i386/cxx14.
Running on pcepsft11.dyndns.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build
See console output.

Errors:

  • [2021-06-18T10:00:58.846Z] FAILED: tree/ntuple/G__ROOTNTuple.cxx lib/ROOTNTuple.pcm
  • [2021-06-18T10:01:00.092Z] /home/sftnight/build/workspace/root-pullrequests-build/build/include/ROOT/RNTupleOptions.hxx:90:58: error: 'const' qualifier may not appear after the virtual specifier 'override'

@phsft-bot
Copy link
Copy Markdown

Build failed on windows10/cxx14.
Running on null:C:\build\workspace\root-pullrequests-build
See console output.

…<RNTupleWriteOptions>`

Using a `std::unique_ptr<>` avoids object slicing in case `fOptions` points to
an instance of a derived class.

a derived
Move the `{Serialize,Deserialize}String` functions to the
`ROOT::Experimental::Internal::RNTupleSerialization` namespace where they may be
reused by other parts of RNTuple.
Use `OC_SX` (distribute among all targets) for OIDs that relate to ntuple
metadata, e.g. header/footer.
Enable customizing the object class for user data OIDs via `RNTupleWriteOptionsDaos`.
…SF` -> `OC_SX`

The `OC_SX` object class (i.e. distribute data among all targets) is probably
a more reasonable default.  This parameter can be changed per-ntuple via
`RNTupleWriteOptionsDaos`.
@jalopezg-git jalopezg-git force-pushed the ntuple-daos-oclass-impl branch from d34d798 to 3fe5bbf Compare June 18, 2021 16:05
@phsft-bot
Copy link
Copy Markdown

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

@jalopezg-git jalopezg-git merged commit 14b8e83 into root-project:master Jun 18, 2021
@jalopezg-git jalopezg-git deleted the ntuple-daos-oclass-impl branch June 18, 2021 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ntuple, daos] Enable user customization of DAOS object class

4 participants