Skip to content

[ntuple] RClassField: add support to load/store class hierarchies#8552

Merged
jalopezg-git merged 6 commits intoroot-project:masterfrom
jalopezg-git:ntuple-tclass-hierarchy
Sep 29, 2021
Merged

[ntuple] RClassField: add support to load/store class hierarchies#8552
jalopezg-git merged 6 commits intoroot-project:masterfrom
jalopezg-git:ntuple-tclass-hierarchy

Conversation

@jalopezg-git
Copy link
Copy Markdown
Contributor

This pull-request extends the support in RClassField to load/store arbitrary types by correctly handling inheritance. See below for the list of changes.

Changes or fixes:

  • An internal subfield named :XXX is created for each inherited XXX class; additional fields are recursively generated for data members/base classes.
  • Reading/writing an object of the derived class includes both, direct and inherited data members. Non-persistent data members, i.e. //!, are not stored.

This PR closes issue #7856.

@phsft-bot
Copy link
Copy Markdown

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

@jalopezg-git jalopezg-git linked an issue Jun 28, 2021 that may be closed by this pull request
@phsft-bot
Copy link
Copy Markdown

Build failed on mac11.0/cxx17.
Running on macphsft20.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Errors:

  • [2021-06-28T13:02:00.724Z] FAILED: tree/ntuple/G__ROOTNTuple.cxx lib/ROOTNTuple.pcm
  • [2021-06-28T13:02:01.293Z] /Users/sftnight/build/workspace/root-pullrequests-build/build/include/ROOT/RField.hxx:301:33: error: constexpr variable cannot have non-literal type 'const std::string' (aka 'const basic_string<char, char_traits<char>, allocator<char> >')

@phsft-bot
Copy link
Copy Markdown

Build failed on ROOT-debian10-i386/cxx14.
Running on pcepsft11.dyndns.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build
See console output.

Errors:

  • [2021-06-28T13:25:49.843Z] FAILED: tree/ntuple/G__ROOTNTuple.cxx lib/ROOTNTuple.pcm
  • [2021-06-28T13:25:50.778Z] /home/sftnight/build/workspace/root-pullrequests-build/build/include/ROOT/RField.hxx:301:33: error: constexpr variable cannot have non-literal type 'const std::string' (aka 'const basic_string<char>')

@phsft-bot
Copy link
Copy Markdown

Build failed on windows10/cxx14.
Running on null:C:\build\workspace\root-pullrequests-build
See console output.

@jalopezg-git jalopezg-git force-pushed the ntuple-tclass-hierarchy branch from 7666f9b to 0f23836 Compare June 28, 2021 16:25
@phsft-bot
Copy link
Copy Markdown

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

@jalopezg-git jalopezg-git force-pushed the ntuple-tclass-hierarchy branch from 0f23836 to bc8e03b Compare July 2, 2021 09:22
@phsft-bot
Copy link
Copy Markdown

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

@phsft-bot
Copy link
Copy Markdown

Build failed on ROOT-debian10-i386/cxx14.
Running on pcepsft11.dyndns.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build
See console output.

Failing tests:

Copy link
Copy Markdown
Contributor

@jblomer jblomer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! Let's add a few more tests to that:

  • Test skipping non-persistent members
  • Test the most complicated (artificial) example for a class hierarchy, e.g. diamond inheritance and members that have types of the inherited classes
  • Test a class with a non-trivial constructor (we probably have to think about construction more carefully)
  • Trying to read/write polymorphic classes: perhaps we can detect with TClass if we have a derived class in our hands. In which case... I guess we want to warn? Or silently slice?

@phsft-bot
Copy link
Copy Markdown

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

@phsft-bot
Copy link
Copy Markdown

Build failed on ROOT-debian10-i386/cxx14.
Running on pcepsft10.dyndns.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Errors:

  • [2021-09-26T01:30:18.891Z] stderr: error: Failed to merge in the changes.
  • [2021-09-26T01:30:23.898Z] CMake Error at /home/sftnight/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1045 (message):

@phsft-bot
Copy link
Copy Markdown

Build failed on ROOT-performance-centos8-multicore/default.
Running on olbdw-01.cern.ch:/data/sftnight/workspace/root-pullrequests-build
See console output.

Errors:

  • [2021-09-26T01:30:29.931Z] CMake Error at /data/sftnight/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1045 (message):

@phsft-bot
Copy link
Copy Markdown

Build failed on windows10/cxx14.
Running on null:C:\build\workspace\root-pullrequests-build
See console output.

Errors:

  • [2021-09-26T01:30:03.742Z] stderr: error: Failed to merge in the changes.
  • [2021-09-26T01:30:30.276Z] CMake Error at C:/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1045 (message):

@phsft-bot
Copy link
Copy Markdown

Build failed on mac11.0/cxx17.
Running on macphsft20.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Errors:

  • [2021-09-26T01:30:39.389Z] CMake Error at /Users/sftnight/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1045 (message):

@phsft-bot
Copy link
Copy Markdown

Build failed on ROOT-ubuntu16/nortcxxmod.
Running on sft-ubuntu-1604-1.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Errors:

  • [2021-09-26T01:30:34.057Z] stderr: error: Failed to merge in the changes.
  • [2021-09-26T01:30:42.605Z] CMake Error at /mnt/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1045 (message):

@phsft-bot
Copy link
Copy Markdown

Build failed on mac1014/python3.
Running on macitois21.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Errors:

  • [2021-09-26T01:33:54.490Z] stderr: error: Failed to merge in the changes.
  • [2021-09-26T01:34:04.242Z] CMake Error at /Volumes/HD2/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1045 (message):

@jalopezg-git jalopezg-git force-pushed the ntuple-tclass-hierarchy branch from ef435b4 to 97c7b5d Compare September 26, 2021 02:10
@phsft-bot
Copy link
Copy Markdown

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

@phsft-bot
Copy link
Copy Markdown

Build failed on mac11.0/cxx17.
Running on macphsft20.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Failing tests:

Improve support of storage of arbitrary types via TClass by handling also
members inherited from base classes. A subfield named `:XXX` is created for each
inherited XXX class.

As per a discussion in a previous I/O meeting, always use `CaptureValue()` in
`ReadGlobalImpl()` (so that reading a new entry is roughly equal to copy
assignment).
… classes

Use `Detail::RFieldBase::Create()` instead, as base classes might not be handled
by `RClassField`, e.g. we might have `std::vector<T>` as a base class.
Test storage of types that inherit from a templated base class, e.g.
```
template <class T>
struct Foobar : public std::vector<T> {
   // ...
};
```
@jalopezg-git jalopezg-git force-pushed the ntuple-tclass-hierarchy branch from 97c7b5d to 10a3e0b Compare September 26, 2021 18:14
@phsft-bot
Copy link
Copy Markdown

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

@jalopezg-git
Copy link
Copy Markdown
Contributor Author

  • Test skipping non-persistent members

Done. See check in test ntuple_types.cxx, line 155.

  • Test the most complicated (artificial) example for a class hierarchy, e.g. diamond inheritance and members that have types of the inherited classes

Done. Also, I have added a test for types inheriting from a templated class, based on the mails exchanged with Marcin Nowak.

  • Trying to read/write polymorphic classes: perhaps we can detect with TClass if we have a derived class in our hands. In which case... I guess we want to warn? Or silently slice?

Currently, we throw -see test ntuple_types.cxx:165-. For a future PR, maybe we can change this behavior.

@phsft-bot
Copy link
Copy Markdown

Build failed on mac11.0/cxx17.
Running on macphsft20.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Failing tests:


for (auto baseClass : ROOT::Detail::TRangeStaticCast<TBaseClass>(*fClass->GetListOfBases())) {
TClass *c = baseClass->GetClassPointer();
auto subField = Detail::RFieldBase::Create(std::string(kPrefixInherited) + c->GetName(),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think for the moment it's fine to use the type name as field name. Together with the v1 serialization, I'd like to go through the RField classes and change the use of type names altogether to generic names (e.g. R_0, R_1, etc.). This way, field names are valid C++ identifiers (except for the : of base classes).

Copy link
Copy Markdown
Contributor

@jblomer jblomer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent! Ready to go!

@jalopezg-git jalopezg-git merged commit c502d3a into root-project:master Sep 29, 2021
@jalopezg-git jalopezg-git deleted the ntuple-tclass-hierarchy branch September 29, 2021 09:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ntuple] Add support for storing class hierarchies in RNTuple

4 participants