Skip to content

Performance regression in rasterio.merge merge when moving from rasterio1.3.10 to 1.4.1 #3228

@swesemeyer

Description

@swesemeyer

Expected behavior and actual behavior.

rasterio.merge merge is very slow in 1.4.1 compared to 1.3.10
The attached script shows the following timing when merging 52 tiny tifs

with 1.3.10 we have

gdal version: 3.8.4
rasterio version: 1.3.10
python version: 3.10.15 (main, Sep 9 2024, 00:00:00) [GCC 14.2.1 20240801 (Red Hat 14.2.1-1)]
os version: Linux-6.11.4-201.fc40.x86_64-x86_64-with-glibc2.39
starting merge with rasterio 1.3.10
merging took 0.03301119804382324 seconds

while when using 1.4.1

gdal version: 3.9.2
rasterio version: 1.4.1
python version: 3.10.15 (main, Sep 9 2024, 00:00:00) [GCC 14.2.1 20240801 (Red Hat 14.2.1-1)]
os version: Linux-6.11.4-201.fc40.x86_64-x86_64-with-glibc2.39
starting merge with rasterio 1.4.1
merging took 4.2044456005096436 seconds

Steps to reproduce the problem.

please see attached script for an example
I trace the main performance hit to the following call in merge (line 431):

                temp_src = src.read(
                    out_shape=temp_shape,
                    window=src_window,
                    boundless=True,
                    masked=True,
                    indexes=indexes,
                    resampling=resampling,
                )

This call takes much longer in 1.4.1. So I believe the DatasetReader is the main culprit.

Environment Information

see above

Installation Method

run the attached script it
Regression.zip

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions