Repository navigation
Make PyYAML a strict dependency of astropy #11626
Description
Description
The yaml library (provided by PyYAML) is central to the serialization functionality used for losslessly storing Table data to ECSV, FITS and HDF5 formats. The ECSV format in particular is seeing fairly wide usage, and is now supported in the StarLink java library STIL and TOPCAT. To this point we have accepted having yaml as an optional dependency, but this means that it cannot be used in any required parts of astropy.
A recent example was to improve the storage of cosmological model parameters. This would nicely be done as an ECSV table, but the yaml dependence was a stopper.
While we do not intend to go down a slippery slope of adding many dependencies, it seems like yaml is justified and would enable future enhancements to astropy core using ECSV or YAML-based storage of data within the package itself.
From the Python packaging perspective this is a small package and should not measurably change the package footprint for a typical astropy installation.
Activity
mhvk commentedon Apr 22, 2021
I'm in favour. I feel any additional dependency needs to have a clear logic, and pyyaml satisfies that: numpy for array-based computation, erfa for astronomy specific calculations, and pyyaml to allow round-trip storage.
I think that if we stick to requiring similarly broad use for any new dependencies, the risk of adding too many remains small. I do think it may be useful to have a "recommended" set of possible additional packages (scipy, matplotlib being the obvious ones).
mwcraig commentedon Apr 22, 2021
I'm in favor of this. It is a pretty light-weight dependency and seems relevant to several subpackages.
kabartay commentedon Apr 23, 2021
I'm in favor of this. It has a lot of advantages that can be later beneficial for Astropy package.
astrofrog commentedon Apr 23, 2021
I’m in favour of this
taldcroft commentedon Apr 23, 2021
@pllim or other infrastructure wizard - so far we have all agreement on this idea. How to make this a reality with a PR? I'm not sure where are all the places that need update. My first guess would be searching the repo for pyerfa and use that as a starting point. It shows up 32 places, which isn't too bad??
It would be really sweet to get this for 4.3. 😸
Cadair commentedon Apr 23, 2021
pyyaml can happily build from an sdist without compiled extensions right?
pllim commentedon Apr 23, 2021
Can we wait till 5.0? Introducing a new required dependency a week before feature freeze is kinda last-minute?
taldcroft commentedon Apr 23, 2021
@pllim - OK, maybe I got over-excited.. I guess 5.0 would make sense.
pllim commentedon Apr 23, 2021
@taldcroft , probably want to send out a memo to astropy-dev mailing list too...
taldcroft commentedon Apr 23, 2021
It does use Cython. I don't see any obvious other extensions but I'm not entirely sure.
There is the subtlety that libyaml is highly recommended along with PyYAML for performance, but it runs just fine (according to the docs) without it.
taldcroft commentedon Apr 23, 2021
And of course there are the doc tests, which are currently done without yaml. No matter how hard I try I still mess this up, like just now in #11569. There are a bunch of doc tests that are skipped because of yaml, so I'll be glad to fix that.
pllim commentedon Apr 23, 2021
Pretty sure we have a test job that runs with all optional dependencies installed, but I can see the problem with core functionality not being tested in all the jobs.
embray commentedon Apr 26, 2021
👍 Also PyYAML is pretty commonly installed as a dependency of other packages.