Skip to content

fix(nanoevents): read RNTuple field descriptions as docstrings in dask mode - #1646

Merged
lgray merged 3 commits into
scikit-hep:masterfrom
Thanatos9404:fix-rntuple-field-doc
Oct 26, 2026
Merged

lgray merged 3 commits into
scikit-hep:masterfrom
Thanatos9404:fix-rntuple-field-doc

Conversation

@Thanatos9404

Copy link
Copy Markdown
Contributor

Part of #1641 (item 2, dask mode).

Problem

Reading any RNTuple with mode="dask" fails as soon as something is computed, including the flat samples in tests/samples:

from coffea.nanoevents import NanoAODSchema, NanoEventsFactory

events = NanoEventsFactory.from_root(
    {"tests/samples/nano_dy_rntuple.root": "Events"},
    schemaclass=NanoAODSchema,
    mode="dask",
).events()
events.Muon.pt.compute()
  File ".../coffea/nanoevents/factory.py", line 204, in load_buffers
  File ".../uproot/behaviors/RNTuple.py", line 821, in arrays
  File ".../uproot/behaviors/RNTuple.py", line 605, in to_akform
  File ".../uproot/models/RNTuple.py", line 588, in field_form
AttributeError: 'RField' object has no attribute 'title'

Root cause

NanoEventsFactory.from_root passes ak_add_doc={"__doc__": "title", "typename": "typename"} to uproot.dask. When uproot builds the form it already reads description instead of title for RNTuple fields, so building the graph works. At compute time, though, _map_schema_uproot.load_buffers passes the same mapping straight to tree.arrays. For an RNTuple that ends up in field_form, which looks up RField.title, and RNTuple fields only have a description.

Change

load_buffers now replaces "title" with "description" in the mapping when the tree is an RNTuple, the same substitution uproot.dask makes when it builds the form. It is the same title/description split that eager and virtual mode already make in UprootSourceMapping._extract_base_form. TTrees get the mapping unchanged, and a boolean ak_add_doc is passed through as before.

Tests

test_nanoaod_schema and test_pfnano_schema in tests/test_nanoevents_rntuple.py now also run in dask mode. They compute the RNTuple and the TTree versions of each sample and compare them, like the eager and virtual cases do. test_base_schema is unchanged, since its treemaker case has nested fields (see below).

Without the fix, the four new dask cases fail with the AttributeError above:

python -m pytest tests/test_nanoevents_rntuple.py -k dask
4 failed, 20 deselected

With the fix:

python -m pytest tests/test_nanoevents_rntuple.py
22 passed, 2 xfailed

python -m pytest tests/test_nanoevents.py tests/test_nanoevents_rntuple.py tests/test_nanoevents_treemaker.py tests/test_nanoevents_pfnano.py tests/test_nanoevents_delphes.py tests/test_nanoevents_physlite.py tests/test_nanoevents_edm4hep.py tests/test_nanoevents_auto.py -m "not dask_client" -n 4
329 passed, 10 skipped, 2 xfailed

python -m pytest (same files) -m "dask_client"
3 passed, 341 deselected

pre-commit run --files src/coffea/nanoevents/factory.py tests/test_nanoevents_rntuple.py
all hooks passed

A short script also confirmed that dask mode now returns the same events as eager mode for the flat RNTuple samples (nano_dy, nano_dimuon, nano_tree with NanoAODSchema, pfnano with PFNanoAODSchema, nano_dy_empty), and that the field parameters differ between dask and eager in the same way for both versions of nano_dy: in dask mode the same 379 of 1499 fields carry __doc__ and typename parameters that eager mode leaves out, for the RNTuple and for the TTree.

Run on Windows with Python 3.14, uproot 5.7.7, awkward 2.14.0, dask-awkward 2026.2.1.

Not covered

Nested RNTuple fields in dask mode still do not work, which the issue already attributes partly to uproot. With the versions above, computing a nested field of tests/samples/treemaker_rntuple.root in dask mode (events.Electrons) had not finished after 100 seconds, and the whole record had not finished after several minutes. The time goes into dask-awkward's column optimization (buffer_keys_required_to_compute_shapes, called from uproot's necessary_columns) before any data is read. Computing the whole record of treemaker.root in dask mode takes about 3 seconds. Item 1 of the issue (NanoAOD behaviors for nested fields) is also left for a separate change.

Thanatos9404 and others added 2 commits October 15, 2026 03:16
…k mode

In dask mode, from_root passes ak_add_doc={"__doc__": "title", ...} to
uproot.dask. uproot already reads the description instead of the title
for RNTuple fields when it builds the form, but load_buffers passed the
same mapping straight to tree.arrays, and an RNTuple field has no title,
so every RNTuple read in dask mode failed with
AttributeError: 'RField' object has no attribute 'title'.

load_buffers now asks an RNTuple for "description" wherever the mapping
asks for "title", the same substitution uproot.dask makes. TTrees are
read exactly as before.

The RNTuple NanoAOD and PFNano tests now also run in dask mode and
compare against the TTree samples.
@lgray

lgray commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator

Thanks for the fix @Thanatos9404 - a quick pass looks reasonable. I'm letting the bot have a pass at this, if there's nothing major I'll merge it.

@read-the-docs-community

read-the-docs-community Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Documentation build overview

📚 coffea-hep | 🛠️ Build #35399450 | 📁 Comparing d3ce64e against latest (a9ebaab)

  🔍 Preview build  

2 files changed
± api/coffea.jetmet_tools.CorrectionLibJERSF.html
± modules/coffea.jetmet_tools.html

@lgray

lgray commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator

oh - for the future if you're sloptalking please preface it with something that says so. People like know the difference between interacting with a bot and a human.

@lgray lgray left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI text below 🤖

Thanks for this. It also unblocks RNTuple reads in graphed mode (coffea#1637), which share this load_buffers.

Review notes, with fixes pushed as a follow-up commit:

  • load_buffers can request no docstrings at all. Only buffers leave this mapping. Docstrings come from the form uproot.dask builds, which already maps title to description for RNTuple. Checked: with ak_add_doc=False here, a TTree's computed Muon.pt.__doc__ is still 'pt'. So ak_add_doc=False fixes the crash with less code and less work per partition.
  • The root cause is in uproot. HasFields.arrays / field_form read the attribute named in ak_add_doc directly, so plain uproot.dask({"nano_dy_rntuple.root": "Events"}, ak_add_doc={"__doc__": "title"}).compute() raises the same AttributeError with no coffea involved. That is worth an uproot issue; this PR stays justified either way.

Comment thread src/coffea/nanoevents/factory.py Outdated
# genuinely-required ("!load") branch that is absent still raises
# loudly at buffer-access time.
present_keys = [key for key in keys if key in tree]
ak_add_doc = interp_options["ak_add_doc"]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI text below 🤖

The docstrings this remap produces are discarded: only buffers leave PreloadedSourceMapping, and __doc__ on the result comes from the meta form. Probe: forcing ak_add_doc=False here leaves the TTree's Muon.pt.__doc__ == 'pt' in dask lazy/computed and virtual mode, and fixes the RNTuple 'RField' object has no attribute 'title' crash in both dask and graphed mode. Suggest replacing the block with ak_add_doc=False and a short comment saying why.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI text below 🤖

Fixed in d3ce64e: ak_add_doc=False with a two-line comment replaces the remap.

elif mode == "eager":
assert ttree.layout.is_all_materialized
assert rntuple.layout.is_all_materialized
elif mode == "dask":

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI text below 🤖

These dask legs catch the crash (all 4 fail with the AttributeError when the factory change is reverted), but they don't test the docstring claim: check_parameters=False ignores __doc__, and every RNTuple sample's field descriptions are empty ('' in dask, computed and virtual mode). A docstring assertion would need a sample RNTuple with non-empty descriptions. Not changed in the follow-up commit.

Only buffers leave the mapping; docstrings come from the form uproot.dask builds, and RNTuple fields have no title.

Assisted-by: ClaudeCode:claude-opus-5-5
@lgray
lgray enabled auto-merge (squash) October 25, 2026 23:48
@lgray
lgray merged commit 28a5a74 into scikit-hep:master Oct 26, 2026
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants