[RNTuple] Support Intel DAOS as storage backend#6825
[RNTuple] Support Intel DAOS as storage backend#6825jblomer merged 12 commits intoroot-project:masterfrom
Conversation
jalopezg-git
force-pushed
the
jalopezg-rntuple-daos
branch
from
fc0fd51 to
4049033
Compare
|
Starting build on |
|
Build failed on ROOT-fedora30/cxx14. Errors:
|
|
Build failed on ROOT-debian10-i386/cxx14. Errors:
|
|
Build failed on mac1015/cxx17. Errors:
|
jalopezg-git
force-pushed
the
jalopezg-rntuple-daos
branch
from
4049033 to
232cb9c
Compare
|
Starting build on |
|
Build failed on ROOT-fedora30/cxx14. Errors:
|
tree/ntuple/CMakeLists.txt
Outdated
| ROOTVecOps | ||
| ) | ||
|
|
||
| # TODO(jalopezg): make daos support optional |
There was a problem hiding this comment.
Hey @jalopezg-r00t I recomend you here to use find_package(DAOS) https://github.com/HDFGroup/vol-daos/blob/master/src/CMakeLists.txt#L18, but please add it SearchInstalledSoftware.cmake (as an example you can use uring section -> https://github.com/root-project/root/blob/master/cmake/modules/SearchInstalledSoftware.cmake#L1173-L1190). Then you will need to add https://github.com/HDFGroup/vol-daos/blob/master/CMake/FindDAOS.cmake here: https://github.com/root-project/root/tree/master/cmake/modules/. Then you will not need to hardcode anything 👍 even while testing backend on Openlab nodes.
There was a problem hiding this comment.
Thanks, @oshadura! I took that shortcut as we needed to present some results for the Intel DAOS User Group meeting last week. Thanks for the information, it will save me some time! :-)
|
Build failed on ROOT-debian10-i386/cxx14. Errors:
|
|
Build failed on mac1015/cxx17. Errors:
|
|
Build failed on windows10/cxx14. Errors:
|
jalopezg-git
force-pushed
the
jalopezg-rntuple-daos
branch
from
232cb9c to
c5467cf
Compare
|
Starting build on |
|
Build failed on ROOT-fedora30/cxx14. Errors:
|
|
Build failed on mac1015/cxx17. Errors:
|
|
Build failed on ROOT-debian10-i386/cxx14. Errors:
|
|
Build failed on windows10/cxx14. Errors:
|
Can we not use the usual approach? If daos is not turned off, look for daos, leave daos on if found. If not found, turn it off, unless fail-on-missing where we'd error out. |
jalopezg-git
force-pushed
the
jalopezg-rntuple-daos
branch
from
6566c69 to
f779c1c
Compare
|
Starting build on |
|
Build failed on mac11.0/cxx17. Failing tests: |
|
@jalopezg-r00t Please rebase to the latest master |
oshadura
left a comment
oshadura
left a comment
There was a problem hiding this comment.
LGTM from CMake point of view (I just added small suggestion/comment)
…nkDaos via a 'daos://' URI
…ont_open` Try to create the container before the `daos_cont_open()` call. The `daos_cont_create()` call should not fail if the container already exists.
…o translation unit
libdaos_mock provides an implementation for a small subset of libdaos that enables in-memory storage. This is especially interesting for testing environments that do not have access to a DAOS deployment.
As suggested in the code review, we demote the `RDaosObject` class template in favor of simpler code. Therefore, we will only support `std::uint64_t` as dkey/akey. This is a separate commit so that we can recover the old templated version if need be.
jalopezg-git
force-pushed
the
jalopezg-rntuple-daos
branch
from
f779c1c to
a8b3422
Compare
|
Starting build on |
6 participants
The changes in this pull request allows RNTuple to use Intel DAOS as storage backend (through libdaos). This support is still experimental.
Currently, each page is assigned a sequential OID at the time it is commited; reserved OIDs are used to store the header, the footer, and the anchor. Each DAOS container cannot contain more than one ntuple.
We expect at least three different follow-up pull requests that will address the following: