Writing a COG to a file-like object in "wb" mode fails in 1.4a2 #3064
Description
Expected behavior and actual behavior.
Expected: writing a COG to a Python file opened in "wb" mode or a BytesIO object works, as it does in 1.3.9
Actual: the following error is thrown in 1.4a1 and 1.4a2
ERROR 10: Pointer 'hSrcDS' is NULL in 'GDALCreateCopy'.
Traceback (most recent call last):
File "/home/danielevans/repositories/image-processing/rio_bytes.py", line 33, in <module>
with rasterio.open(f2, mode="w", **profile_cog) as ds:
File "rasterio/_base.pyx", line 450, in rasterio._base.DatasetBase.__exit__
File "rasterio/_base.pyx", line 441, in rasterio._base.DatasetBase.close
File "/usr/lib/python3.10/contextlib.py", line 584, in close
self.__exit__(None, None, None)
File "/usr/lib/python3.10/contextlib.py", line 576, in __exit__
raise exc_details[1]
File "/usr/lib/python3.10/contextlib.py", line 561, in __exit__
if cb(*exc_details):
File "/home/danielevans/.cache/pypoetry/virtualenvs/image-processing-LWpwzOMN-py3.10/lib/python3.10/site-packages/rasterio/io.py", line 168, in __exit__
self._env.close()
File "/usr/lib/python3.10/contextlib.py", line 584, in close
self.__exit__(None, None, None)
File "/usr/lib/python3.10/contextlib.py", line 576, in __exit__
raise exc_details[1]
File "/usr/lib/python3.10/contextlib.py", line 561, in __exit__
if cb(*exc_details):
File "rasterio/_base.pyx", line 450, in rasterio._base.DatasetBase.__exit__
File "rasterio/_base.pyx", line 439, in rasterio._base.DatasetBase.close
File "rasterio/_io.pyx", line 2400, in rasterio._io.BufferedDatasetWriterBase.stop
File "rasterio/_err.pyx", line 359, in rasterio._err.exc_wrap_pointer
rasterio._err.ObjectNullError: Pointer 'hSrcDS' is NULL in 'GDALCreateCopy'.
Our test suite writes out the same image to a bunch of other output types (file paths, rasterio Memfiles, S3, ...) that don't fail.
Steps to reproduce the problem.
Script below. Weirdly, it works for the plain GTiff profile, but the COG kicks out an error.
import rasterio
import numpy as np
from affine import Affine
data = np.zeros((100, 100))
profile_gtiff = {
"width": data.shape[0],
"height": data.shape[1],
"count": 1,
"dtype": np.uint8,
"transform": Affine(3.5, 0.0, 558838.0, 0.0, -3.5, 5927362.0),
"crs": 32630,
"nodata": 0,
"driver": "GTiff",
"compress": "DEFLATE",
}
profile_cog = {
**profile_gtiff,
"driver": "COG",
}
print("Write GTiff")
with open("foo.tiff", "wb") as f1:
with rasterio.open(f1, mode="w", **profile_gtiff) as ds:
ds.write(data, 1)
print("OK")
print("Write COG")
with open("bar.tiff", "wb") as f2:
with rasterio.open(f2, mode="w", **profile_cog) as ds:
ds.write(data, 1)
print("OK")
Environment Information
rasterio info:
rasterio: 1.4a2
GDAL: 3.8.4
PROJ: 9.3.1
GEOS: 3.11.1
PROJ DATA: /home/danielevans/.cache/pypoetry/virtualenvs/image-processing-LWpwzOMN-py3.10/lib/python3.10/site-packages/rasterio/proj_data
GDAL DATA: /home/danielevans/.cache/pypoetry/virtualenvs/image-processing-LWpwzOMN-py3.10/lib/python3.10/site-packages/rasterio/gdal_data
System:
python: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
executable: /home/danielevans/.cache/pypoetry/virtualenvs/image-processing-LWpwzOMN-py3.10/bin/python
machine: Linux-5.15.146.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
Python deps:
affine: 2.4.0
attrs: 23.2.0
certifi: 2024.02.02
click: 8.1.7
cligj: 0.7.2
cython: 3.0.10
numpy: 1.26.4
click-plugins: None
setuptools: 69.2.0
Installation Method
Pip wheels for 1.3.9, 1.4a1, 1.4a2