Skip to content

Make astropy.utils.decorators.wraps much more like functools.wraps - #7554

Merged
pllim merged 10 commits into
astropy:masterfrom
Cadair:wraps_controversial
Apr 20, 2019
Merged

pllim merged 10 commits into
astropy:masterfrom
Cadair:wraps_controversial

Conversation

@Cadair

@Cadair Cadair commented Jun 11, 2018

Copy link
Copy Markdown
Member

Here is a controversial fix to #7551. This removes two features of astropy.utils.decorators.wraps the exclude_args method and the way it overwrote the functions name.

In return for this, we make use of the new __signature__ attribute in Python 3 and we probably don't need a lot more code that I have currently deleted in here.

It seems that the only place astropy was using exclude_args was in the test runner, which is now irrelevant as my changes to that mean it generates the documentation automatically anyway. And my opinion on the name thing is that it's deeply horrific and probably never should have been done in the first place 😉

@astropy-bot

astropy-bot Bot commented Jun 11, 2018 •

Copy link
Copy Markdown

Check results are now reported in the status checks at the bottom of this page.

@Cadair

Cadair commented Jun 11, 2018

Copy link
Copy Markdown
Member Author

It's also worth noting, that there is only one other use of make_function_with_signature in astropy, in modeling. Which could probably be replaced with direct manipulation of the Python 3 signature machinery. So we could probably remove all of that as well.

@Cadair

Cadair commented Jun 11, 2018

Copy link
Copy Markdown
Member Author

It appears there is one quirk with supports_nddata which I don't understand, but I am not going to waste my time debugging it until people are actually on board with this plan.

@astrofrog

Copy link
Copy Markdown
Member

Ok so just to be clear, wraps in Python 3.5 now no longer requires us to have our own implementation? If so, that's fine by me in principle

@bsipocz bsipocz added the utils label Jun 11, 2018
@bsipocz bsipocz added this to the v3.1 milestone Jun 11, 2018
Comment thread astropy/modeling/core.py Resolved
@Cadair

Cadair commented Jun 11, 2018

Copy link
Copy Markdown
Member Author

@astrofrog Yes, but not quite with perfect feature parity.

@pllim

pllim commented Jun 11, 2018

Copy link
Copy Markdown
Member

I am not going to waste my time debugging it

Looks like a "catch 22" situation. I am not convinced until tests can pass. 😉

@Cadair

Cadair commented Jun 11, 2018

Copy link
Copy Markdown
Member Author

haha, the question is, are we willing to loose the exclude_args and name trickery for the sake of this refactor? If so then I will try and make it work. Obviously if I can't make it work it's just a waste of everyones time...

@pllim

pllim commented Jun 11, 2018

Copy link
Copy Markdown
Member

I cannot comment on the nddata stuff but maybe @mwcraig or @MSeifert04 can?

@mwcraig

mwcraig commented Jun 11, 2018

Copy link
Copy Markdown
Member

I’m gong to be offline for the next week, can look when I’m back

@mhvk

mhvk commented Jun 11, 2018

Copy link
Copy Markdown
Contributor

Very much in favour from the peanut gallery! Though since it is actually in the documentation, I think wraps also would need to be deprecated first (as far as I'm concerned, this can be by returning the python standard version with a deprecation warning that it will not stay in astropy.utils.decorators).

@bsipocz

bsipocz commented Jun 12, 2018

Copy link
Copy Markdown
Member

I didn't find a single place on github where this was used, so we most probably can get away with @mhvk's suggestion for a very quick deprecation period.

@Cadair

Cadair commented Jun 12, 2018

Copy link
Copy Markdown
Member Author

@bsipocz did you also check for make_function_with_signature by chance? I assume we should deprecate that for a little while as well?

@bsipocz

bsipocz commented Jun 12, 2018

Copy link
Copy Markdown
Member

@Cadair - There is minimal use for that, and most of the hits seems to be in copies of astropy itself: https://github.com/search?q=make_function_with_signature&type=Code

I've only found one genuine use of it in one of Perry's repos, so my guess is the usage is indeed fairly low.

@MSeifert04

MSeifert04 commented Jun 30, 2018 •

Copy link
Copy Markdown
Contributor

I like it - provided we don't completely break support_nddata :)

@pllim pllim modified the milestones: v3.1, v3.2 Oct 26, 2018
@pllim

pllim commented Oct 26, 2018

Copy link
Copy Markdown
Member

Too controversial to merge so close to freeze time, so re-milestoned. 😬

@Cadair

Cadair commented Oct 26, 2018

Copy link
Copy Markdown
Member Author

I think this is one for the meeting in Dec.

@pllim

pllim commented Oct 26, 2018

Copy link
Copy Markdown
Member

meeting in Dec

OK, added to breakout planning board.

@astropy-bot

This comment has been minimized.

@mhvk

This comment has been minimized.

@pllim

This comment has been minimized.

@mhvk mhvk added the keep-open label Nov 15, 2018
@mhvk

mhvk commented Nov 15, 2018

Copy link
Copy Markdown
Contributor

OK, added "keep open". I think there was general consensus that this was a good idea as long as support_nddata keeps working.

@bsipocz

bsipocz commented Nov 15, 2018

Copy link
Copy Markdown
Member

@Cadair - Could you please rather rebase this that keeping it alive by the label? IMO we shouldn't over use the label otherwise easily end up it the previous situation of having long abandoned PRs all over.

@mhvk

mhvk commented Nov 15, 2018

Copy link
Copy Markdown
Contributor

@bsipocz - that was the fault of me and @pllim - but point taken that a rebase would be better.

@mwcraig
mwcraig force-pushed the wraps_controversial branch from e756e93 to 8061a16 Compare April 19, 2019 19:13
@codecov

codecov Bot commented Apr 19, 2019 •

Copy link
Copy Markdown

Codecov Report

Merging #7554 into master will increase coverage by 0.02%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #7554      +/-   ##
==========================================
+ Coverage    86.9%   86.93%   +0.02%     
==========================================
  Files         393      397       +4     
  Lines       58584    59123     +539     
  Branches     1064     1100      +36     
==========================================
+ Hits        50914    51397     +483     
- Misses       7051     7085      +34     
- Partials      619      641      +22
Impacted Files Coverage Δ
astropy/modeling/core.py 90.21% <ø> (+0.68%) ⬆️
astropy/utils/codegen.py 90.9% <100%> (-1.28%) ⬇️
astropy/timeseries/core.py 97.56% <100%> (ø) ⬆️
astropy/nddata/decorators.py 97.64% <100%> (+0.02%) ⬆️
astropy/io/fits/util.py 83.16% <100%> (ø) ⬆️
astropy/utils/decorators.py 93.8% <100%> (ø) ⬆️
astropy/tests/runner.py 77.62% <100%> (+0.1%) ⬆️
astropy/config/paths.py 64.51% <100%> (ø) ⬆️
astropy/utils/metadata.py 97.22% <100%> (ø) ⬆️
astropy/stats/lombscargle/__init__.py 75% <0%> (-25%) ⬇️
... and 65 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d1f2fd5...387cab3. Read the comment docs.

@mwcraig

mwcraig commented Apr 19, 2019

Copy link
Copy Markdown
Member

Waiting to see if doc issue still occurs. It does not locally (though I'm seeing an error in an unrelated part of the docs)

@pllim

pllim commented Apr 19, 2019

Copy link
Copy Markdown
Member

Doc still failed. FYI

Warning, treated as error:
toctree references unknown document 'api/astropy.utils.decorators.wraps'
Sphinx Documentation subprocess failed with return code 2
Exited with code 2

@mwcraig

mwcraig commented Apr 19, 2019

Copy link
Copy Markdown
Member

Doc still failed. FYI

Yeah, still can't reproduce locally, which is making it hard to troubleshoot....

@mwcraig

mwcraig commented Apr 19, 2019

Copy link
Copy Markdown
Member

OK, reproduced and fixed (hopefully). The other changes in the last push were to replace imports of astropy.utils.wraps with functools.wraps so that we can eventually not provide wraps at all in astropy.utils.

@mwcraig mwcraig left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this @Cadair; I made a couple minor changes, think it is ready to go.

@mwcraig mwcraig added the zzz 💤 merge-when-ci-passes Do not use: We have auto-merge option now. label Apr 19, 2019
@Cadair

Cadair commented Apr 23, 2019

Copy link
Copy Markdown
Member Author

Awesome!! Thanks @mwcraig

@bsipocz

bsipocz commented Apr 24, 2019

Copy link
Copy Markdown
Member

Since the deprecation from this PR has been reverted in #8624, I'm adding the no-changelog-entry-needed label to stop the release script from complaining.

@bsipocz

bsipocz commented Jun 28, 2019

Copy link
Copy Markdown
Member

For the sake of the exercise I'm looking at recent benchmark regressions and weirdly this came up as a reason for a 1.3-1.5x slowdown in the convolution benchmarks.
http://www.astropy.org/astropy-benchmarks/#regressions?sort=1&dir=desc

@pllim

pllim commented Jun 28, 2019

Copy link
Copy Markdown
Member

Hmm... Do you think this is because it is using modeling and modeling is using the deprecated code?

new_call = make_function_with_signature(

@bsipocz

bsipocz commented Jun 28, 2019

Copy link
Copy Markdown
Member

Yes, I think it's modeling related, but the regression should have show up in the deprecating PR if it's because of that. I suspect just there are some overheads here that we didn't think about, and/or some non optimal usage of modeling in convolution.

I would say that we need to keep an eye on those benchmarks after the refactoring PR is merged in. Or, do you @astrofrog think that we can run the benchmarks on the PR to check the performance implications? (though either way the refactoring will be merge, so maybe it's not urgent to check it in the PR stage).

pllim added a commit to pllim/astropy that referenced this pull request Dec 30, 2021
since astropy wraps was removed in astropy#7554
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants