You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
AsyncGroup.create_array() got an unexpected keyword argument 'compression' in netcdf3.py module #534
I'm trying to create fsspec files for netCDF 3 datasets on a local filesystem, but am encountering an error during the json reference file creation process:
AsyncGroup.create_array() got an unexpected keyword argument 'compression'
The error is coming from the netCDF3.py module NetCDF3toZarr translate method in the following block:
compression is not included in the create_dataset parameter list according to the zarr documentation, but compressor is, so fixing the argument in this and other create_dataset calls should presumably solve the issue.
Steps to reproduce:
def write_fsspec(fs_read, input_file, output_dir):
with fs_read.open(input_file) as infile:
print(f"Running kerchunk generation for {input_file}...")
chunks = kerchunk.netCDF3.NetCDF3ToZarr(infile, inline_threshold=300)
file_name = os.path.basename(input_file)
file_name = file_name.replace('.nc', '.json')
out_file_name = output_dir + '/' + file_name
with open(out_file_name, "wb") as f:
f.write(json.dumps(chunks.translate()).encode()) # call to netCDF3.py originates here
print(f"Finished writing {out_file_name}")
return out_file_name
dir_path = config['input_dir']
dir_path += '**/*.nc'
file_paths = glob.glob(dir_path, recursive=True)
fs_read = fsspec.filesystem('local')
temp_dir = TemporaryDirectory(prefix=config['output_dir'])
output_files = [write_fsspec(fs_read, f, temp_dir.name) for f in file_paths]
What version of zarr do you have? You may need to downgrade to <3.0. We know that there is some work needed in this repo to make it compatible with the newely-released major version of zarr.
@martindurant Downgrading to Zarr v2.18.4 fixed the issue, though I am now encountering other errors. I'm not sure if these are fsspec or file metadata problems (dataset does not adhere to cmip conventions), but I'll open a different issue if it's something else in fsspec.
Activity
martindurant commentedon Jan 13, 2025
What version of zarr do you have? You may need to downgrade to <3.0. We know that there is some work needed in this repo to make it compatible with the newely-released major version of zarr.
wrongkindofdoctor commentedon Jan 13, 2025
@martindurant Zarr is v3.0.0. I will try downgrading and report back.
wrongkindofdoctor commentedon Jan 13, 2025
@martindurant Downgrading to Zarr v2.18.4 fixed the issue, though I am now encountering other errors. I'm not sure if these are fsspec or file metadata problems (dataset does not adhere to cmip conventions), but I'll open a different issue if it's something else in fsspec.