CI: enable python dependencies caching - #18835

Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

• Do the proposed changes actually accomplish desired goals?
• Do the proposed changes follow the Astropy coding guidelines?
• Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
• Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
• Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see instructions for rebase and squash.
• Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
• Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
• Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
• At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

Welcome to Astropy 👋 and congratulations on your first pull
request! 🎉

A project member will respond to you as soon as possible; in the
meantime, please have a look over the Checklist for Contributed
Code
and make sure you've addressed as many of the questions there as
possible.

If you feel that this pull request has not been responded to in a
timely manner, please send a message directly to the
development mailing
list. If the issue is
urgent or sensitive in nature (e.g., a security vulnerability)
please send an e-mail directly to the private e-mail
[email protected].

👋 Thank you for your draft pull request! Do you know that you can use [ci skip] or [skip ci] in your commit messages to skip running continuous integration tests until you are ready?

As I suspected, this doesn't seem to work: with jobs competing for a single cache key, only one of them is able to save a cache, which in turn will be essentially useless on re-runs since the bulk of resources (mostly bandwidth) we're trying to save are taken up by platform-specific binaries (extensions from numpy, scipy ...), so, I'm not sure that we can cache anything efficiently with the current interface from OpenAstronomy/github-actions-workflows

I'll report this upstream

reported as OpenAstronomy/github-actions-workflows#324

Maybe possible: OpenAstronomy/github-actions-workflows#324 (comment)

Indeed, and apparently I already did the work ! I just forgot to check the logs and undraft. I did now !

I don't think the test failure caused by a failed file download is related, but with dependency caching it's hard to tell.

Note that I'm intentionally not enabling cache for unstable CI, so we can trust the failure isn't related.

I want a couple of other devs to review as well and requested them. If you don't hear back in a week, feel free to ping me and then maybe we can just merge at that point. Thanks.

Let's merge this, it seems to be working. If it all goes to hell the revert button is not far away.