Repository navigation
fix(nanoevents): read RNTuple field descriptions as docstrings in dask mode - #1646
Conversation
…k mode
In dask mode, from_root passes ak_add_doc={"__doc__": "title", ...} to
uproot.dask. uproot already reads the description instead of the title
for RNTuple fields when it builds the form, but load_buffers passed the
same mapping straight to tree.arrays, and an RNTuple field has no title,
so every RNTuple read in dask mode failed with
AttributeError: 'RField' object has no attribute 'title'.
load_buffers now asks an RNTuple for "description" wherever the mapping
asks for "title", the same substitution uproot.dask makes. TTrees are
read exactly as before.
The RNTuple NanoAOD and PFNano tests now also run in dask mode and
compare against the TTree samples.
|
Thanks for the fix @Thanatos9404 - a quick pass looks reasonable. I'm letting the bot have a pass at this, if there's nothing major I'll merge it. |
Documentation build overview
2 files changed± api/coffea.jetmet_tools.CorrectionLibJERSF.html± modules/coffea.jetmet_tools.html |
|
oh - for the future if you're sloptalking please preface it with something that says so. People like know the difference between interacting with a bot and a human. |
lgray
left a comment
There was a problem hiding this comment.
🤖 AI text below 🤖
Thanks for this. It also unblocks RNTuple reads in graphed mode (coffea#1637), which share this load_buffers.
Review notes, with fixes pushed as a follow-up commit:
load_bufferscan request no docstrings at all. Only buffers leave this mapping. Docstrings come from the formuproot.daskbuilds, which already mapstitletodescriptionfor RNTuple. Checked: withak_add_doc=Falsehere, a TTree's computedMuon.pt.__doc__is still'pt'. Soak_add_doc=Falsefixes the crash with less code and less work per partition.- The root cause is in uproot.
HasFields.arrays/field_formread the attribute named inak_add_docdirectly, so plainuproot.dask({"nano_dy_rntuple.root": "Events"}, ak_add_doc={"__doc__": "title"}).compute()raises the sameAttributeErrorwith no coffea involved. That is worth an uproot issue; this PR stays justified either way.
| # genuinely-required ("!load") branch that is absent still raises | ||
| # loudly at buffer-access time. | ||
| present_keys = [key for key in keys if key in tree] | ||
| ak_add_doc = interp_options["ak_add_doc"] |
There was a problem hiding this comment.
🤖 AI text below 🤖
The docstrings this remap produces are discarded: only buffers leave PreloadedSourceMapping, and __doc__ on the result comes from the meta form. Probe: forcing ak_add_doc=False here leaves the TTree's Muon.pt.__doc__ == 'pt' in dask lazy/computed and virtual mode, and fixes the RNTuple 'RField' object has no attribute 'title' crash in both dask and graphed mode. Suggest replacing the block with ak_add_doc=False and a short comment saying why.
There was a problem hiding this comment.
🤖 AI text below 🤖
Fixed in d3ce64e: ak_add_doc=False with a two-line comment replaces the remap.
| elif mode == "eager": | ||
| assert ttree.layout.is_all_materialized | ||
| assert rntuple.layout.is_all_materialized | ||
| elif mode == "dask": |
There was a problem hiding this comment.
🤖 AI text below 🤖
These dask legs catch the crash (all 4 fail with the AttributeError when the factory change is reverted), but they don't test the docstring claim: check_parameters=False ignores __doc__, and every RNTuple sample's field descriptions are empty ('' in dask, computed and virtual mode). A docstring assertion would need a sample RNTuple with non-empty descriptions. Not changed in the follow-up commit.
Only buffers leave the mapping; docstrings come from the form uproot.dask builds, and RNTuple fields have no title. Assisted-by: ClaudeCode:claude-opus-5-5
Part of #1641 (item 2, dask mode).
Problem
Reading any RNTuple with
mode="dask"fails as soon as something is computed, including the flat samples intests/samples:Root cause
NanoEventsFactory.from_rootpassesak_add_doc={"__doc__": "title", "typename": "typename"}touproot.dask. When uproot builds the form it already readsdescriptioninstead oftitlefor RNTuple fields, so building the graph works. At compute time, though,_map_schema_uproot.load_bufferspasses the same mapping straight totree.arrays. For an RNTuple that ends up infield_form, which looks upRField.title, and RNTuple fields only have adescription.Change
load_buffersnow replaces"title"with"description"in the mapping when the tree is an RNTuple, the same substitutionuproot.daskmakes when it builds the form. It is the same title/description split that eager and virtual mode already make inUprootSourceMapping._extract_base_form. TTrees get the mapping unchanged, and a booleanak_add_docis passed through as before.Tests
test_nanoaod_schemaandtest_pfnano_schemaintests/test_nanoevents_rntuple.pynow also run in dask mode. They compute the RNTuple and the TTree versions of each sample and compare them, like the eager and virtual cases do.test_base_schemais unchanged, since its treemaker case has nested fields (see below).Without the fix, the four new dask cases fail with the
AttributeErrorabove:With the fix:
A short script also confirmed that dask mode now returns the same events as eager mode for the flat RNTuple samples (
nano_dy,nano_dimuon,nano_treewithNanoAODSchema,pfnanowithPFNanoAODSchema,nano_dy_empty), and that the field parameters differ between dask and eager in the same way for both versions ofnano_dy: in dask mode the same 379 of 1499 fields carry__doc__andtypenameparameters that eager mode leaves out, for the RNTuple and for the TTree.Run on Windows with Python 3.14, uproot 5.7.7, awkward 2.14.0, dask-awkward 2026.2.1.
Not covered
Nested RNTuple fields in dask mode still do not work, which the issue already attributes partly to uproot. With the versions above, computing a nested field of
tests/samples/treemaker_rntuple.rootin dask mode (events.Electrons) had not finished after 100 seconds, and the whole record had not finished after several minutes. The time goes into dask-awkward's column optimization (buffer_keys_required_to_compute_shapes, called from uproot'snecessary_columns) before any data is read. Computing the whole record oftreemaker.rootin dask mode takes about 3 seconds. Item 1 of the issue (NanoAOD behaviors for nested fields) is also left for a separate change.