Skip to content

Comments

Added driver format argument to _delete_dataset_if_exists#3366

Merged
snowman2 merged 3 commits intorasterio:mainfrom
normanb:nb/openex_format
Dec 11, 2025
Merged

Added driver format argument to _delete_dataset_if_exists#3366
snowman2 merged 3 commits intorasterio:mainfrom
normanb:nb/openex_format

Conversation

@normanb
Copy link
Contributor

@normanb normanb commented Jun 27, 2025

A short PR to improve performance and solve an issue with deleting datasets prior to write.

_delete_dataset_if_exists currently calls open_dataset which triggers a driver scan within GDAL. If the user has specified a format then this should be used to check if the dataset exists. A driver scan also causes some performance overhead.

Not specifying a format can cause a different exception to be thrown which is then not caught in _delete_dataset_if_exists.

An example (I will link the PR) is the support for vsiaz with the GDAL TileDB driver. Creating an array with vsiaz directly, e.g. /vsiaz/myarray will fail if the format is not specified as the driver scan will throw an error as other drivers will be tried. It is not possible to download a portion of /vsiaz/myarray as it is not a single file and hence the checks for headers and file extensions do not apply in GDAL Identify.

@sgillies
Copy link
Member

sgillies commented Jun 28, 2025

Sounds good to me, @normanb ! I don't know what's up with the failing tests in test_rio_transform.py.

@normanb
Copy link
Contributor Author

normanb commented Jun 30, 2025

Thanks @sgillies! I have recreated the issue with test_rio_transform.py

micromamba create -n rasterio_env -c conda-forge python=3.10 gdal=3.9.3 "numpy<2" pytest boto3
micromamba activate rasterio_env

# against rasterio master
pip install -e . --no-binary=rasterio


$ cat << EOF | rio transform --dst-crs EPSG:32618 --precision 2
> [-78.0, 23.0]
> EOF
[192457.13, 2546667.68]

So rio transform is working, however

python -m pytest tests/test_rio_transform.py::test_transform

Fails as per the test suite

    def test_transform(runner):
        """Coordinates are transformed."""
        result = runner.invoke(
            main_group,
            ["transform", "--dst-crs=EPSG:32618", "--precision=2"],
            input="[-78.0, 23.0]",
        )
>       assert result.output.rstrip("\n") == "[192457.13, 2546667.68]"
E       AssertionError: assert '\nAborted!' == '[192457.13, 2546667.68]'
E         
E         - [192457.13, 2546667.68]
E         + 
E         + Aborted!

I will dig into it.

@normanb
Copy link
Contributor Author

normanb commented Jun 30, 2025

Freezing the click CLI runner seemed to fix some bugs e.g. tests/test_rio_transform.py but still leaves some. I will try and run the github actions locally and see if this can be resolved.

@normanb
Copy link
Contributor Author

normanb commented Jun 30, 2025

I ran

act -W '.github/workflows/tests.yaml'

And only got one failing test which didn't match the failures above.

From looking at other PRs it isn't just this PR that broke the CI?

How do you want to progress? It seems like fixing the CI is a PR by itself. Any ideas on things I should try?

@sgillies
Copy link
Member

sgillies commented Jul 1, 2025

@normanb Yes, the workflows are decomposing. I would let them go if I were you. I hope that I'll get some paid time to work on them, some day.

@snowman2 snowman2 added this to the 1.5.0 milestone Sep 11, 2025
@snowman2 snowman2 requested a review from sgillies December 10, 2025 11:44
snowman2
snowman2 approved these changes Dec 10, 2025
View reviewed changes
Copy link
Member

@snowman2 snowman2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @normanb 👍

@snowman2 snowman2 merged commit 1d8ce1d into rasterio:main Dec 11, 2025
26 checks passed
@snowman2
Copy link
Member

snowman2 commented Dec 11, 2025

Superseded by #3473

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@snowman2 snowman2 snowman2 approved these changes

@sgillies sgillies Awaiting requested review from sgillies

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

1.5.0

Development

Successfully merging this pull request may close these issues.

3 participants

@normanb @sgillies @snowman2