-
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Description
What is your issue?
From xarray 2025.9.1 onwards it seems that netcdf4 is not the default engine anymore, but h5netcdf. I thought the PR was #10755, but this was merged 1 day after the 2025.9.1 release. Either way, with this xarray version I see that the default engine is different. I am running into troubles with opening netcdf files from an opendap server. When not setting the engine, there are warnings for h5netcdf and scipy, so I deliberately set it in this example:
import xarray as xr
file_nc_bathy = r'https://opendap.deltares.nl/thredds/dodsC/opendap/deltares/Delft3D/netcdf_example_files/GEBCO_2022/GEBCO_2022_coarsefac08.nc'
data_bathy = xr.open_dataset(file_nc_bathy, engine='h5netcdf')This raises "FileNotFoundError: https://opendap.deltares.nl/thredds/dodsC/opendap/deltares/Delft3D/netcdf_example_files/GEBCO_2022/GEBCO_2022_coarsefac08.nc". When I use engine='netcdf4'. If aiohttp is not installed I get "ImportError: HTTPFileSystem requires "requests" and "aiohttp" to be installed" (but installing it still gives the FileNotFoundError).
I am not sure whether this is a bug and if so, in which package exactly. I would expect that this new default engine, one should also be able to open files on opendap. Although, it seems from pytroll/satpy#786 (comment) that this might not be the case.
The thing is that with the new default engine, users get confusing errors like ImportError and FileNotFoundError. Or if they do not manually set the engine (most of the cases), they get warnings:
C:\Users\veenstra\AppData\Local\miniforge3\envs\dfm_tools_env\Lib\site-packages\xarray\backends\api.py:577: RuntimeWarning: 'h5netcdf' fails while guessing
engine = plugins.guess_engine(filename_or_obj)
C:\Users\veenstra\AppData\Local\miniforge3\envs\dfm_tools_env\Lib\site-packages\xarray\backends\api.py:577: RuntimeWarning: 'scipy' fails while guessing
engine = plugins.guess_engine(filename_or_obj)
Furthermore, in my github tests, I get the ImportError without explicitly setting the engine: Deltares/dfm_tools#1265
Proposed solution
In #10755 there is an exception built in for nczarr:
nczarr_mode = isinstance(path_or_file, str) and path_or_file.endswith(
"#mode=nczarr"
)
if nczarr_mode:
candidates[:] = ["netcdf4"]Would it also be possible to build this in for opendap, so users do not get potentially confusing errors/warnings?
Additional request
Furthermore, I did notice less memory consumption but much more time consumption with h5netcdf in the past: Deltares/dfm_tools#484. I had hoped that the h5netcdf performance issues (h5netcdf/h5netcdf#195 and h5netcdf/h5netcdf#251) would have been resolved before h5netcdf was chosen as the default xarray engine. Would it still be possible to do so?