Repository navigation
Feature Request: astropy.utils.data downloader using requests in urllib2 #1123
Description
For a variety of reasons, but most pressing being a need for cookies 🍪 🍪 🍪, we decided to push towards using requests in place of urllib+urllib2 in astroquery. Unfortunately, this means a different approach than currently implemented in astropy.utils.data.
A progressbar should be possible (e.g. https://gist.github.com/gschizas/3731992) but the caching system may be somewhat more complex.
Activity
eteq commentedon May 29, 2013
I looked at this a bit, and I don't think it should be too hard... Easier in some ways, because a fair amount of
astropy.utils.datais working around oddurllib/urllib2quirks. The catch is the problem discussed in kennethreitz/requests#1364 (which is actually a urllib3 issue). Once that's addressed, I would be game for addingrequeststoexternand starting this transition.In the meantime, @keflavich, you may want to try to structure the
astroqueryimports so that you can easily useastropy.extern.requestswhenever it actually starts to exist.eteq commentedon May 29, 2013
See also #1035 (which I only now noticed).
keflavich commentedon May 29, 2013
Thanks @eteq, I also hadn't noticed #1035.
When you say "structure the imports...", is there anything I need to be careful about? Would it be anything more than a global replace all of
import requestswithfrom astropy.extern import requests?eteq commentedon May 30, 2013
Yeah, that's probably all that would be necessary, now that you mention it. I was thinking you could do some sort of trick of doing like
from astroquery.extern import requests(e.g. add anexternthat just imports requests or something), but a global find-and-replace is probably easier. (Although you also have to dofrom requests import ..., but that should also be easy.)keflavich commentedon Aug 23, 2013
I don't think any changes to astropy are needed at this point, so I'm closing this issue.