Skip to content

Upgrade extern PLY version to 3.9 - #5526

Merged
mhvk merged 4 commits into
astropy:masterfrom
bsipocz:extern_update_ply_3.9
Dec 5, 2016
Merged

mhvk merged 4 commits into
astropy:masterfrom
bsipocz:extern_update_ply_3.9

Conversation

@bsipocz

@bsipocz bsipocz commented Nov 30, 2016

Copy link
Copy Markdown
Member

Note this PR updates to exactly have the PLY released version and thus also reverts the six usage changes from #5259 (assuming that the policy is to keep the external packages unchanged whenever possible).

@mhvk

mhvk commented Nov 30, 2016

Copy link
Copy Markdown
Contributor

The errors in both travis and appveyor are puzzling. Not sure if they are red herrings or not. With the new version, should one ensure that new lextab/parsetab get made? (e.g., in astropy/units/formats)?

@bsipocz

bsipocz commented Nov 30, 2016

Copy link
Copy Markdown
Member Author

Travis is very much puzzling as it passed on my fork here: https://travis-ci.org/bsipocz/astropy/builds/179894615 (except osx that I cancelled due to the traffic jam).
I'll try to look at the details a bit later today or tomorrow.

@pllim pllim added this to the v1.3.0 milestone Nov 30, 2016
@mhvk

mhvk commented Nov 30, 2016

Copy link
Copy Markdown
Contributor

@bsipocz - I decided to restart the failed travis jobs, just in case they were due to some glitch. Though especially the one with coverage enabled looks very suspicious, as it has an internal error with Angle parsing that could easily be due to generating a new parser with ply.

... actually, the appveyor error on python3 is the same as that failed coverage job, so this is almost certainly related. Regenerating the unit format parsers may actually be needed.
... indeed, looking at the previous update (#4003), I see that the various lextab and parsetab were included....

@bsipocz
bsipocz force-pushed the extern_update_ply_3.9 branch from 4db67e6 to 1b1ad79 Compare December 2, 2016 18:00
@bsipocz

bsipocz commented Dec 2, 2016

Copy link
Copy Markdown
Member Author

Good catch @mhvk, now I've updated the parsing tables, too.

Comment thread astropy/extern/ply/lex.py
import os
import inspect

from ...extern.six.moves import zip

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, haven't noticed this was an external package when I added this. Could you re-add this?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or should I rather make a request on PLY itself?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's better to add this upstream, especially that this is purely stylistic change, they have a try/except workaround.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you mean by try / except workaround?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

L45 below, they look for the python version for handling the strings, and for the zip it's not necessary either.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree it's not necessary - but faster and more memory-efficient. 😄

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still, I would rather follow the policy to keep the external packages untouched here, but it would be great if you could contribute any improvements upstream.

import base64
import warnings

from ...extern.six.moves import zip

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here.

@bsipocz

bsipocz commented Dec 5, 2016

Copy link
Copy Markdown
Member Author

@eteq, @astrofrog - Travis failures are due to the glitch on Friday, so this should be good to go, too.

@mhvk mhvk added the external PRs and issues related to external packages vendored with Astropy (astropy.extern) label Dec 5, 2016
@mhvk

mhvk commented Dec 5, 2016

Copy link
Copy Markdown
Contributor

This looks all OK, so will merge. I also made a new "external" label for it...

@mhvk
mhvk merged commit 4acfbc4 into astropy:master Dec 5, 2016
@taldcroft

Copy link
Copy Markdown
Member

I just rebased #5486 on master, but something is odd. When I run python setup.py test this appears to regenerate astropy/units/format/generic_lextab.py and astropy/units/format/generic_parstab.py for 3.9 and I then see diffs in those files (which would appear to be the 3.8 ones in master). Possibly user-error on my part, but not obvious right now what is happening.

@mhvk

mhvk commented Dec 6, 2016

Copy link
Copy Markdown
Contributor

Oops -- I think those should have been included here. @bsipocz - would you be able to do a quick PR with those?? Thanks...

@bsipocz

bsipocz commented Dec 6, 2016

Copy link
Copy Markdown
Member Author

@taldcroft @mhvk - This is weird as I generated them the same way as the other tables. Also a bit weird why it picks up the __version__ and not __tabversion__.
PR will come shortly.

@bsipocz

bsipocz commented Dec 6, 2016

Copy link
Copy Markdown
Member Author

I'm afraid I can't reproduce it, running the tests on master doesn't generate any new files for me. Now running git status on travis to see whether these are generated there.

@bsipocz

bsipocz commented Dec 6, 2016

Copy link
Copy Markdown
Member Author

@taldcroft - By any chance you run the tests with python2? I see the tables generated on travis only on the two python2 builds.

@taldcroft

Copy link
Copy Markdown
Member

Yes, it was on Python 2.

saimn added a commit to saimn/astropy that referenced this pull request Jan 23, 2017
Ref astropy#5581, fixes astropy#5726.

After astropy#5526 updated PLY to 3.9, some files were re-generated at runtime
(astropy/coordinates/{angle_lextab.py,angle_parsetab.py}). This is because of
this line https://github.com/dabeaz/ply/blob/master/ply/lex.py#L901 which
assumes that the lextab parameter in `lex.lex` is a str only, not unicode, which
leads to some surprise when unicode_literals is used on Python 2 !

So this commit just wraps the lextab value to make sure that a str is always
passed to `lex.lex`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external PRs and issues related to external packages vendored with Astropy (astropy.extern)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants