rasterio 1.3.8 fails to read a raster from AWS S3 bucket #2888
Description
Expected behavior and actual behavior.
- the expected behavior is that the process successfully reads the raster from the S3 bucket
- the actual behavior is that the process crashes with an error and fails to do the expected
Steps to reproduce the problem.
Here is a test raster (zipped). You can upload it on a bucket and try for yourself: test.tif.zip
This MVE works on 1.3.7
import rasterio
from fs_s3fs import S3FS
bucket_name = '<bucket_name>'
bucket_path = '<bucket_path>'
raster_name = '<raster_name>'
fs = S3FS(
bucket_name=bucket_name,
dir_path=bucket_path,
aws_access_key_id=aws_access_key_id,
aws_secret_access_key=aws_secret_access_key,
aws_session_token=aws_session_token,
)
with rasterio.Env(), fs.openbin(raster_name, "r") as file_handle:
with rasterio.open(file_handle) as src:
data = src.read()but on 1.3.8 it crashes with the following stacktrace: error.txt
Another example which works for both versions if one usesAWSSession from rasterio.session
import rasterio
from rasterio.session import AWSSession
bucket_name = '<bucket_name>'
bucket_path = '<bucket_path>'
raster_name = '<raster_name>'
session = AWSSession(
aws_access_key_id=aws_access_key_id,
aws_secret_access_key=aws_secret_access_key,
aws_session_token=aws_session_token
)
with rasterio.Env(session=session):
with rasterio.open(f's3://{bucket_name}/{bucket_path}/{raster_name}') as src:
data = src.read()Environment Information
rasterio info:
rasterio: 1.3.8
GDAL: 3.6.4
PROJ: 9.0.1
GEOS: 0.0.0
PROJ DATA: /Users/mlubej/.pyenv/versions/eo-learn/lib/python3.10/site-packages/rasterio/proj_data
GDAL DATA: /Users/mlubej/.pyenv/versions/eo-learn/lib/python3.10/site-packages/rasterio/gdal_data
System:
python: 3.10.4 (main, Nov 30 2022, 16:56:30) [Clang 14.0.0 (clang-1400.0.29.202)]
executable: /Users/mlubej/.pyenv/versions/eo-learn/bin/python
machine: macOS-13.5-x86_64-i386-64bit
Python deps:
affine: 2.4.0
attrs: 23.1.0
certifi: 2023.07.22
click: 8.1.6
cligj: 0.7.2
cython: None
numpy: 1.25.1
snuggs: 1.4.7
click-plugins: None
setuptools: 58.1.0
Installation Method
installed with pip install rasterio -U using pip==23.2.1