Repository navigation
Deprecate use of chararray in io.fits #3862
Description
As raised in #3854 and earlier issues, Numpy is gradually trying to deprecate and eventually kill off the old Numarray chararray class. This is fine, but io.fits still uses it for FITS tables. This is especially important in supporting FITS ASCII tables, for its ability to automatically rstrip() individual strings returned from the array.
However, for FITS data we can just use our own class that has this capability (we don't need any of the other chararray methods). It might be nice to go ahead and use some version of my encoded_text_array class for FITS string columns (and possible string columns in other Astropy tables), and to add optional rstrip support to that class (it already has automatic rstripping, but that should probably be optional).
I think for backwards compat the new class should inherit chararray methods, but they would all raise deprecation warnings. Similarly isinstance(..., chararray) checks should raise a deprecation warning (that the class will no longer be a chararray subclass after long).
Activity
astrofrog commentedon Sep 29, 2015
Don't think this will happen for 1.1 so removing milestone
pllim commentedon Feb 14, 2024
Hmm did numpy 2.0 already forced our hands in this issue (maybe #15363 ) or not?
neutrinoceros commentedon Feb 14, 2024
@pllim I don't think so. In fact, numpy 2.0 has received some attention in that area. For instance:
Besides,
np.chararrayis still very much used inio.fits, so when the time comes, I'm sure we'll know 😅mhvk commentedon Feb 14, 2024
I think the idea is still that we'd slowly stop using
np.charand instead directly use functions from the newnp.strings. But there is no specific deprecation planned for 2.0 (and the ufuncs are now used insidechararrayso we'll benefit from the speed-up).saimn commentedon Jan 26, 2026
Seems to be happening on Numpy side: #19216
We need to find a solution since currently io.fits relies on chararray to automatically strip trailing spaces from string columns.
mhvk commentedon Jan 26, 2026
If we only want to keep the
.rstrip(), it may be easiest to define our ownndarraysubclass, overriding__getitem__like done inchararray: https://github.com/numpy/numpy/blob/f590d096927d9165979d3a7fb3b72a07364344f2/numpy/_core/defchararray.py#L595-L599Maybe at least for a test? One could add something that calls other overridden methods but with a deprecation warning.
The alternative would, I think, mean creating a fresh array in which strings are pre-stripped, i.e., a copy (possibly one could check whether it would make a difference first; not implausible that many FITS columns are fully filled, eg., because they just contain "Y" or "N").
saimn commentedon Jan 27, 2026
Yes I think we need to keep the rstrip behavior, and ideally support the char methods with a deprecation warning.
I was going to say that the first option is better but actually, as we already have a conversion mechanism (and cache) for some column types (unsigned integer, boolean), maybe it's not unreasonable to do the rstrip that way. In the end it's pretty much the same as what chararray does.
mhvk commentedon Jan 27, 2026
True, though with
chararrayyou only do it as needed, so the memory footprint doesn't double. But perhaps with a check of whether the.rstrip()is in fact needed it might be the simplest...pllim commentedon Feb 2, 2026
The deprecation warning is bubbling downstream in packages that pull in both numpy-dev and astropy-dev and turn unhandled warnings into errors.
Do we have a patch that can be implemented relatively quickly? What is the way forward besides max-pinning numpy here? Worst case is we copy over all the deprecated chararray stuff from numpy but that is probably not elegant.
saimn commentedon Feb 4, 2026
One problem is that we don't know yet if
np.char.arrayis also going to be deprecated (in the same Numpy release). If this change lands in Numpy dev then the current PR is useless.Then there is the question of what to do long term: ndarray subclass implementing the on the fly rstrip, or a simple ndarray and rstrip is done with the conversion mechanism already present in io.fits. I have some local changes for the second approach, almost working, currently that would be my preference.
This would be a breaking change, which we could do in 8.0, but would be nice to find a way to warn people that chararray method will disappear.
So maybe something we could do now (and release in a 7.2.x version) would be to make a chararray subclass, suppressing the warning from Numpy, and raising a warning when chararray methods are called ?
neutrinoceros commentedon Feb 4, 2026
I disagree. The existing PR, if nothing else, would allow us to see any other failure modes currently hidden in devdeps jobs.
Something like #19241 ?
saimn commentedon Feb 4, 2026
I was thinking something like #19267 to avoid duplicating the whole
chararrayclass.saimn commentedon Feb 4, 2026
One issue is that we don't have much time to deprecate things on our side if we want to remove chararray in 8.0, but I don't know how long Numpy will keep the deprecated class (i.e. can we wait 9.0 ?).
neutrinoceros commentedon Feb 4, 2026
deprecation cycles in numpy are usually about a year long, and often longer. There should be enough time to adjust.
Human gold patch for astropy__astropy-19267