[ntuple] Change to the version 1 RC1 binary format#8897
[ntuple] Change to the version 1 RC1 binary format#8897jalopezg-git merged 73 commits intoroot-project:masterfrom
Conversation
|
Starting build on |
|
Build failed on ROOT-debian10-i386/cxx14. Errors:
|
|
Build failed on mac11.0/cxx17. Failing tests: |
|
Starting build on |
|
Build failed on mac11.0/cxx17. Failing tests: |
|
Starting build on |
|
Build failed on windows10/cxx14. Errors:
|
|
Build failed on mac11.0/cxx17. Failing tests: |
|
Starting build on |
|
Build failed on windows10/cxx14. Errors:
|
| The first 32bit integer references the physical column ID. | ||
| The second 32bit integer references a field that needs to have the "alias field" flag set. | ||
| The ID of the alias column itself is given implicitly by the serialization order. | ||
| In particular, alias columns have larger IDs than physical columns. |
There was a problem hiding this comment.
Can't you intertwine alias field and physical field?
There was a problem hiding this comment.
What is the purpose of having the alias column being on disk (since the alias field is already there)?
There was a problem hiding this comment.
You can make an alias field the child of a physical field. That should be as far as it goes in terms of mixing.
The motivation for alias fields come from the conversion of of TTree files with leaf count branches (e.g. njets plus jet_eta, jet_pt, etc.). This would be translated into an (anonymous) collection in RNTuple but that implies that the field names change, e.g. to jet.eta, jet.pt. To make sure existing RDF analysis code continues working on the converted file, I though it would be useful to remember their original names.
There was a problem hiding this comment.
What I mean is why is the alias represented, on disk, as a column rather than the column being recreated out of the alias field when reading the RNtuple meta-data?
|
Build failed on mac11.0/cxx17. Failing tests: |
Co-authored-by: Philippe Canal <[email protected]>
Co-authored-by: Philippe Canal <[email protected]>
Co-authored-by: Philippe Canal <[email protected]>
Co-authored-by: Philippe Canal <[email protected]>
Co-authored-by: Philippe Canal <[email protected]>
…:GetBitsOnStorage()` (NFC) Co-authored-by: Javier Lopez-Gomez <[email protected]>
Co-authored-by: Javier Lopez-Gomez <[email protected]>
Co-authored-by: Philippe Canal <[email protected]>
…xxx` diagnostic in tests
jalopezg-git
force-pushed
the
ntuple-binary-format-v1
branch
from
51148cc to
10de5d2
Compare
|
Starting build on |
5 participants
This Pull request:
Adds the version 1 binary format specification and upgrades the implementation from version 0 to version 1. This is a backwards incompatible change. As of version 1, RNTuple is supposed to stay backwards compatible.
The new binary format is a precondition for, among other things, the following required features
Compared to the v0 format, the header is ~40% smaller and the footer ~100% smaller (after zstd compression).
This PR sets the pre-release tag 1 in the binary format, so files written in this version trigger a warning on reading. The pre-release tag might increase in follow-up PRs. Once stable, the pre-release tag will be set to 0.
Other follow-up PRs: