Skip to content

Fix convolve_fft_parameters psf_pad and fft_pad, which were being ignored - #4366

Merged
eteq merged 13 commits into
astropy:masterfrom
keflavich:convolve_fft_parameters
Mar 10, 2016
Merged

eteq merged 13 commits into
astropy:masterfrom
keflavich:convolve_fft_parameters

Conversation

@keflavich

Copy link
Copy Markdown
Contributor

The parameters psf_pad and fft_pad were being ignored if boundary!=None,
which was intentional but was opaque to users. Now, errors will be raised if
incompatible parameters are passed and the keywords will be respected when
specified.

psf_pad, fft_pad, and boundary are somewhat redundant, but there is good
reason to include both: boundary controls something "physical", i.e., it determines
what the edge effects will be. psf_pad is closely related to this, and perhaps
is genuinely redundant. However, both psf_pad and fft_pad can seriously affect
the performance of the convolution in non-obvious ways. Therefore, I think they
all deserve to be included.

I have also included as a string a report on the performance of convolve_fft
under different conditions. This came up because a user was complaining they
were waiting forever for a convolution of a 6000x6000 with a 3000x3000 array,
which with the default parameters will create a set of 16384x16384 arrays!

Comment thread astropy/convolution/convolve.py Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To help prevent unnecessary looping over the whole mask arrays, I'd write it as

if (not interpolate_nan and not quite and
    (np.any(nanmasarray) or np.any(nanmaskkernel))):

(python will short-circuit the if-statement; np.any seems more logical than counting).

@eteq eteq added this to the v1.2.0 milestone Dec 2, 2015
@embray

embray commented Dec 2, 2015

Copy link
Copy Markdown
Member

I would consider this a bug fix...

Comment thread astropy/convolution/convolve.py Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace single backticks with double backticks.

@embray

embray commented Dec 2, 2015

Copy link
Copy Markdown
Member

Actually, above comments aside, could you remove the profiling code from this PR? It seems pretty separate. Would be nice to have in a separate change though.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The argument handling updates all make sense--I think they should just be committed as a bug fix.

@keflavich

Copy link
Copy Markdown
Contributor Author

@embray sure, can split this into two. I also agree about the plot instead of table. It probably doesn't belong in the code, that was just quickest. I'll at least separate these tomorrow, but I don't know if I'll get to cleaning up the profiler.

@embray

embray commented Dec 2, 2015

Copy link
Copy Markdown
Member

No hurry on the profiler obviously, but thanks for adding it nonetheless.

@keflavich
keflavich force-pushed the convolve_fft_parameters branch from d0262ad to 7dbb848 Compare December 5, 2015 07:55
@eteq

eteq commented Dec 5, 2015

Copy link
Copy Markdown
Member

@keflavich - can you clarify if this if you consider this an API change? My thinking is that this means that in some cases code that ran before now will instead raise an error. That's probably good going forward, but is still an API change/new feature (and hence belongs in the next major release). Do you agree?

@keflavich

Copy link
Copy Markdown
Contributor Author

@eteq I would still consider this a bugfix rather than an API change. The API change was supposed to happen ~ a year ago, but I missed some of the possible conditions so that it was never actually implemented. The documented API hasn't changed, at least, except to clarify when and whether psf_pad/fft_pad supercede boundary. Just as importantly, code that ran using these parameters (and having them ignored) might have run incorrectly without raising an error when it should have. On the other hand, the most likely failure is that the function consumed more memory than it should have but returned an identical result: so, errors affecting production code are unlikely.

@eteq

eteq commented Dec 8, 2015

Copy link
Copy Markdown
Member

@embray @astrofrog - so what do we want to do about this given @keflavich's statement above? sneak it in for 1.1, or schedule it for 1.1.1? And should it also go into 1.0.x? (My thinking is yes at least to the latter, unsure for the former question)

@eteq

eteq commented Dec 8, 2015

Copy link
Copy Markdown
Member

(Oh, and this is missing a changelog entry, @keflavich - feel free to put it in whichever version you like and then we can always reorganize later when we decide where it should finally go)

@eteq eteq modified the milestones: v1.1.1, v1.2.0 Dec 8, 2015
@eteq

eteq commented Dec 8, 2015

Copy link
Copy Markdown
Member

After thinking a bit (and discussing w/ @astrofrog), it's probably better to have this be in 1.1.1 but not 1.1.0, unless there's some critical fix that ends up requiring a 1.1rc3.

@embray

embray commented Dec 8, 2015

Copy link
Copy Markdown
Member

Right, let's delay this for now, I think.

@embray embray modified the milestones: v1.1.1, v1.1.2 Jan 7, 2016
@eteq eteq modified the milestones: v1.0.9, v1.1.2 Mar 10, 2016
@eteq eteq added the Bug label Mar 10, 2016
@eteq

eteq commented Mar 10, 2016

Copy link
Copy Markdown
Member

Alright, I'm going to go ahead and make the command decision to merge this and backport it to 1.0.9. I'll trust @keflavich's judgement that it's actually a "true" bug.

eteq added a commit that referenced this pull request Mar 10, 2016
Fix convolve_fft_parameters psf_pad and fft_pad, which were being ignored
@eteq
eteq merged commit e51c1a3 into astropy:master Mar 10, 2016
eteq added a commit that referenced this pull request Mar 10, 2016
Fix convolve_fft_parameters psf_pad and fft_pad, which were being ignored
eteq added a commit that referenced this pull request Mar 10, 2016
Fix convolve_fft_parameters psf_pad and fft_pad, which were being ignored
@keflavich
keflavich deleted the convolve_fft_parameters branch March 10, 2016 05:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants