Skip to content

Comments

Add support for signed 8 bit images#1595

Merged
sgillies merged 1 commit intorasterio:maint-1.0from
Kirill888:signedbyte
Feb 8, 2019
Merged

Add support for signed 8 bit images#1595
sgillies merged 1 commit intorasterio:maint-1.0from
Kirill888:signedbyte

Conversation

@Kirill888
Copy link
Contributor

Signed 8-bit images are somewhat "second-class" in GDAL, but it can read/write them nevertheless.

Even though GDAL only has unsigned GDT_Byte raster data type, signed vs unsigned
images can be differentiated via PIXELTYPE property. It is set to SIGNEDBYTE
for signed 8-bit file. This is how GDAL's native python binding compute numpy.dtype from dataset object:

https://github.com/OSGeo/gdal/blob/90dde4a3fc0d89373a62000ae14e424fc41f7306/gdal/swig/python/osgeo/gdal_array.py#L299-L300

Similarly, on output side, setting option PIXELTYPE='SIGNEDBYTE' when creating file with GDT_Byte dtype will generate signed int8 image.

@Kirill888
Copy link
Contributor Author

@sgillies I know that signed 8 bit images are a bit "dumb", but in our case we have a need to read NetCDF CF files, and NetCDF CF convention forbids unsigned integer types, so instead int8 is used for small, usually categorical data. NetCDF also doesn't have masking layer unlike GeoTIFF and so nodata value is often used instead.

Rasterio behaviour of silently casting int8 data into uint8 and then discarding invalid nodata values (those that fall outside of valid range of the data type) makes it impossible for us to read nodata information for int8 NetCDF CF files. We have work-arounds using gdal directly, and should probably use netcdf library instead, but still would be nice to have support for int8 images directly in rasterio.

When you have an opportunity to look into this can you please let me know what your thoughts are on this:

  • not a priority for right now, but agree in principle
  • int8 is too niche to justify adding extra complexity
  • happy to support int8, but would rather do it differently
  • other

Copy link
Member

@sgillies sgillies left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Kirill888 this is great, and just how I would do it. Even better than I would have done it. Would you be willing to base this PR off the maint-1.0 branch so we can get it into 1.0.19?

Sorry for the delay!

Even though GDAL only has unsigned GDT_Byte raster data type, signed vs unsigned
images can be differentiated via PIXELTYPE property. It is set to 'SIGNEDBYTE'
for signed 8-bit images. This is how GDAL's native python binding compute
numpy.dtype from dataset object.
@Kirill888 Kirill888 changed the base branch from master to maint-1.0 February 7, 2019 21:39
@Kirill888
Copy link
Contributor Author

Thanks @sgillies I have rebased to maint-1.0

@rouault
Copy link
Contributor

rouault commented Nov 8, 2022

Cf OSGeo/gdal#6634 "RFC 87: Signed int8 data type for raster" that will likely be implemented in GDAL 3.7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants