Skip to content

Update docs to reflect open_mfdataset default chunk behaviour #10567

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
keewis merged 3 commits into from
Oct 21, 2025
Merged

Update docs to reflect open_mfdataset default chunk behaviour #10567

keewis merged 3 commits into from
Oct 21, 2025

Conversation

@jemmajeffree
Copy link
Contributor

@jemmajeffree jemmajeffree commented Jul 25, 2025

Looking at #5704 and #9038 (and various other issues that reference these), it seems like there's no easy solution and the current open_mfdataset chunk default is going to stay for a while. i.e. chunks=None as a kwarg to open_mfdataset is overwritten with chunks={} before calling open_dataset, so the default chunks are disk chunks not single-file chunks.

xarray/xarray/backends/api.py

Line 1677 in 5ce69b2

open_kwargs = dict(engine=engine, chunks=chunks or {}, **kwargs)

In which case, we should probably update the docs to reflect this current behaviour.

@welcome
Copy link

welcome bot commented Jul 25, 2025

Thank you for opening this pull request! It may take us a few days to respond here, so thank you for being patient.
If you have questions, some answers may be found in our contributing guidelines.

@jemmajeffree
Copy link
Contributor Author

jemmajeffree commented Aug 5, 2025

@dcherian would you mind taking a quick look at this for me?
It's unclear to me while some of the tests are failing, given the only thing I changed was a docstring

@keewis
Copy link
Collaborator

keewis commented Aug 5, 2025 •
edited
Loading

I would expect the numba CI failure to either be unrelated or just flaky (and we don't have to care about the PR labeler here). I've restarted the CI, let's see if that changes things.

Edit: all CI passes!

@jemmajeffree
Copy link
Contributor Author

jemmajeffree commented Aug 5, 2025

Thanks! Anything else I should do here, or am I just waiting for someone to check the docstring update is correct and merge?

@keewis keewis requested a review from dcherian August 6, 2025 12:47
@jemmajeffree
Copy link
Contributor Author

jemmajeffree commented Oct 20, 2025

@keewis @dcherian could I get some eyes on this — anything else I need to do to be able to merge?

We had a couple of people here get confused by these docs the other day

@keewis
Copy link
Collaborator

keewis commented Oct 21, 2025 •
edited
Loading

Thanks for the ping, @jemmajeffree, this completely fell off my radar

I'll go ahead and merge since this is such a small change

@keewis keewis enabled auto-merge (squash) October 21, 2025 07:23
@keewis keewis merged commit 101a5c2 into pydata:main Oct 21, 2025
35 of 36 checks passed
dcherian added a commit to dcherian/xarray that referenced this pull request Oct 21, 2025
@dcherian
Merge branch 'main' into min-concat-mem
954b4ac 
* main:
  Update docs to reflect open_mfdataset default chunk behaviour (pydata#10567)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@keewis keewis keewis approved these changes

@dcherian dcherian Awaiting requested review from dcherian

Assignees

No one assigned

Labels

io topic-backends

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jemmajeffree @keewis