astropy.utils.data.download_file() crashes when show_progress=False #908

mdboom · 2013-03-28T00:17:06Z

This is a follow-up of #865 by @mdboom .

>>> from astropy.utils.data import download_file
>>> download_file('https://github.com/astropy/astropy/blob/master/README.rst')
Downloading https://github.com/astropy/astropy/blob/master/README.rst
|===========================================|  24k/ 24k (100.00%)        00s
'/tmp/tmptcOh_V'

So far so good. But when I try to suppress the progress bar:

>>> download_file('https://github.com/astropy/astropy/blob/master/README.rst',
...               show_progress=False)
ERROR: TypeError: 'unicode' does not have the buffer interface [astropy.utils.console]
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-15-8afb32881abb> in <module>()
----> 1 download_file('https://github.com/astropy/astropy/blob/master/README.rst', show_progress=False)

/home/lim/rhel6/lib/python2.7/site-packages/astropy-0.3.dev3725-py2.7-linux-x86_64.egg/astropy/utils/data.pyc in download_file(remote_url, cache, show_progress)
    881                         if os.path.exists(f.name):
    882                             os.remove(f.name)
--> 883                         raise
    884 
    885         if cache:

/home/lim/rhel6/lib/python2.7/site-packages/astropy-0.3.dev3725-py2.7-linux-x86_64.egg/astropy/utils/console.pyc in __exit__(self, exc_type, exc_value, traceback)
    634 
    635     def __exit__(self, exc_type, exc_value, traceback):
--> 636         return self._obj.__exit__(exc_type, exc_value, traceback)
    637 
    638     def update(self, value):

/home/lim/rhel6/lib/python2.7/site-packages/astropy-0.3.dev3725-py2.7-linux-x86_64.egg/astropy/utils/console.pyc in __exit__(self, exc_type, exc_value, traceback)
    565             color_print(u' [Done]', 'green', file=file)
    566         else:
--> 567             color_print(u' [Failed]', 'red', file=file)
    568         flush()
    569 

/home/lim/rhel6/lib/python2.7/site-packages/astropy-0.3.dev3725-py2.7-linux-x86_64.egg/astropy/utils/console.pyc in color_print(*args, **kwargs)
    159             if isinstance(msg, bytes):
    160                 msg = msg.decode('ascii')
--> 161             write(msg)
    162         write(end)
    163 

TypeError: 'unicode' does not have the buffer interface

pllim · 2013-03-27T20:21:59Z

It comes down to how to handle streams in astropy.utils.console.color_print().

>>> color_print(u' [Failed]', 'red', file=io.BytesIO())
ERROR: TypeError: 'unicode' does not have the buffer interface [astropy.utils.console]
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-10-c54de5862bd0> in <module>()
----> 1 color_print(u' [Failed]', 'red', file=io.BytesIO())

/home/lim/rhel6/lib/python2.7/site-packages/astropy-0.3.dev3725-py2.7-linux-x86_64.egg/astropy/utils/console.pyc in color_print(*args, **kwargs)
    159             if isinstance(msg, bytes):
    160                 msg = msg.decode('ascii')
--> 161             write(msg)
    162         write(end)
    163 

TypeError: 'unicode' does not have the buffer interface

But this is okay:

>>> color_print(u' [Failed]', 'red', file=io.StringIO())

pllim · 2013-03-27T20:50:33Z

p/s: This also breaks #552

…work correctly

mdboom · 2013-03-28T00:17:39Z

Sorry about that -- I think an intermediate version somehow got committed. A fix is attached.

pllim · 2013-03-28T13:34:13Z

@mdboom , looks like Travis failed for doc build. I tried to look at the log but it took forever to load, so I am not sure why it failed.

mdboom · 2013-03-28T14:00:40Z

I think #911 fixes the docbuild -- We just have to get that merged into master (I'm hoping to get it reviewed first) and then rebased over here.

pllim · 2013-04-01T14:37:19Z

@mdboom , since #911 is merged. Is this going to be merged soon?

mdboom · 2013-04-01T15:05:07Z

Sure. Done.

astropy.utils.data.download_file() crashes when show_progress=False

eteq · 2013-04-01T21:28:02Z

It appears this introduced a separate build_sphinx error that was masked by #911? Anyway, the travis builds are failing on master after this.

mdboom · 2013-04-02T12:40:16Z

Hmmm... I can't reproduce any problem with build_sphinx locally, and the log on Travis seems to be truncated (at least for me, it stops in the middle of the package build and doesn't even get to the doc build). Any ideas?

pllim · 2013-04-02T20:01:00Z

I have the latest master on my branch (plus my own additions) and the doc compiles fine on Python 2.7.3 and Sphinx 1.1.2.

eteq · 2013-04-02T21:49:37Z

I also can't reproduce the actual failure locally, either. I also don't see how this could possibly have caused the failure, though, because it doesn't seem to be related to the failure (something about not being able to pickle WCSBase - see below for how I got the log). I'm experimenting right now with Travis on eteq/astropy to see if I can determine the cause.

@mdboom - to get the un-truncated log, click the gear on the Travis log page and choose "download log". For this one, that will get you https://s3.amazonaws.com/archive.travis-ci.org/jobs/5911405/log.txt

They seem to be in the middle of some sort of change to how they handle logs, and it isn't quite working for >10000 line logs.

eteq · 2013-04-02T21:50:51Z

Whoops - I meant https://s3.amazonaws.com/archive.travis-ci.org/jobs/5958983/log.txt

Fix astropy#908, broken by astropy#865. Makes show_progress == False …

5b97878

…work correctly

mdboom added a commit that referenced this pull request Apr 1, 2013

Merge pull request #908 from mdboom/console/hide-download-progress-fix

e464a07

astropy.utils.data.download_file() crashes when show_progress=False

mdboom merged commit e464a07 into astropy:master Apr 1, 2013

This was referenced Apr 2, 2013

travis sphinx build failing due to WCSBase pickling issues #934

Closed

Docs won't build on Sphinx 1.2b1 due to WCSBase #935

Merged

mdboom deleted the console/hide-download-progress-fix branch May 21, 2014 23:54

Uh oh!

astropy.utils.data.download_file() crashes when show_progress=False #908

astropy.utils.data.download_file() crashes when show_progress=False #908

Uh oh!

Conversation

mdboom commented Mar 28, 2013

Uh oh!

pllim commented Mar 27, 2013

Uh oh!

pllim commented Mar 27, 2013

Uh oh!

mdboom commented Mar 28, 2013

Uh oh!

pllim commented Mar 28, 2013

Uh oh!

mdboom commented Mar 28, 2013

Uh oh!

pllim commented Apr 1, 2013

Uh oh!

mdboom commented Apr 1, 2013

Uh oh!

eteq commented Apr 1, 2013

Uh oh!

mdboom commented Apr 2, 2013

Uh oh!

pllim commented Apr 2, 2013

Uh oh!

eteq commented Apr 2, 2013

Uh oh!

eteq commented Apr 2, 2013

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants