You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Per discussion in gh-21277, I agree it would be best to switch the new lombscargle implementation to use floating_mean=False by default because of the disruption in downstream numerical results we are causing. If it were truly fixing a bug it would normally be allowed to cause a "break", but seems like it is more like a general improvement we could issue a warning of some sort about (i.e., warn that the "better" setting will eventually become the default if the consumer doesn't explicitly specify a value for it?).
@mhvk@jakevdp@neutrinoceros do you want us to keep this numerically backward compatible and issue a warning that the default will change in a future release? I believe the argument is that the "new" default is generally better, but I'm not quite sure it passes the requirement of a "bug fix allowing a break" at first glance.
@tylerjereddy I follow your logic. I wouldn't know what the threshold is to "break" now versus later. But I would suspect most would agree that setting floating_mean=True is better in the long run--especially for most "naive" users, that wouldn't even be aware of the downsides to the previous method.
Not sure what is going on, but comments from Jake and others have me concerned so just opening this to make sure the matter is thoroughly considered before cutting a release.
Pasting my comment from #21277 (comment) since it's relevant I think:
One place where fit_mean=True will cause problems is when you use lombscargle to find the window frequency by passing in a unit signal, like this:
signal_psd=lombscargle(x, y, freqs)
window_psd=lombscargle(x, np.ones_like(y), freqs)
If fit_mean is True by default, then window_psd is under-constrained, and you'd have to explicitly set fit_mean=False to get meaningful output. This is not "backward-compatible" even in the looser sense of the term.
(I'll do the same, since I think it's probably worthwhile until we nail down where a single discussion is taking place)
I'm confused. fit_mean is a astropy.lombscargle parameter, and it is true by default. And, as best as I can tell, it is equivalent (in spirit) to floating_mean in scipy.lombscargle, which (currently) is also true by default. So, it looks like to me, the two methods are in agreement that a "floating mean" is best, and should be the default.
Yes, in astropy it has always been set to True, because it's the more sensible default when the option is available.
Here the argument is that changing the default is a breaking change, and so it shouldn't be done quietly because it will break downstream projects (as it did in the linked astropy issue).
I propose we continue the discussion here if there's more to say.
Yes, I was in fact mentioning fit_mean mainly trying to understand what model astropy has been using so far, and seeing that it currently isTrue by default in the main astropy.lombscargle and its other implementations, I was close to proposing that the scipy wrapper should also adopt this as new default, say as of Astropy 7.0. But that would create even more confusion in the form of different behaviour as long as we support Scipy <= 1.14.
So having floating_mean default to False would make our implementation a bit easier as we don't have to check for scipy.version to add a new argument, if needed, but now that we now about it, we can probably live with both.
do you want us to keep this numerically backward compatible and issue a warning that the default will change in a future release?
This would be the ideal solution as far as I can tell. I totally get the argument that changing the default is desirable, but making the change progressive over the course of a deprecation period would be much easier on us downstream than the brutal change we're seeing now.
Thanks a lot for suggesting it first !
So having floating_mean default to False would make our implementation a bit easier as we don't have to check for scipy.version to add a new argument, if needed, but now that we now about it, we can probably live with both.
But, presumably, even if the floating_mean default change was delayed, you'd have to check for the version eventually. Right?
I'm just trying to understand what would be different between breaking it now versus in the future. I think astropy would still need to check the scipy version anyways.
I'm just trying to understand what would be different between breaking it now versus in the future.
I think that's the whole point of deprecation warnings or future warnings -- just to give the ecosystem some time to adapt; there may also be projects with less developer eyes and cross-project testing than astropy, so we tend to be conservative this close to the base of the ecosystem.
Folks can then start to gradually adjust to avoid the warning, and report any surprises to us as they do so, for a more gradual rollout. Otherwise, astropy and/or us may end up cutting more releases due to surprises, etc.
I think there's a good argument here for not changing the default at all. The estimator produces a different statistic depending on the value of fit_mean, sometimes it's the better choice (e.g. with a signal that has an unknown mean) and sometimes it isn't (e.g. the window function signal I mentioned above). No single setting is correct in all situations, and given that I'd lean toward not changing the behavior in order to avoid unnecessary churn downstream. If we were introducing a new algorithm, we might choose a different default. But since the current behavior has existed for probably 20+ years, there's a high bar for changing it, and I'm not convinced the change is warranted.
Ok, that's helpful feedback, I think the needle is moving farther away from justifying a breaking change at any point then. Perhaps the docs can clarify some of the subtleties/tradeoffs of option/algorithm choice if they don't already.
That's a good point on the docs. More than just the parameter default would have to change, as the docstring is written in several places assuming it's True.
However, I'm unsure about how much—or even what—to add to the Notes section (and how long the maintainers are comfortable with it being). Especially since we all agree the True value (in general) is the preferred option (makes me wonder if adding that note to the Parameters section is appropriate).
for what it's worth, I'm +1 on not changing the default behavior at all if it can be avoided.
I'll still note that the following patch should make astropy more robust to an hypothetical future change: astropy/astropy#17163
but of course that doesn't accomplish anything for every other user of this function, so it shouldn't weight too much on the decision to change the default or not.
Activity
tylerjereddy commentedon Oct 11, 2024
@mhvk @jakevdp @neutrinoceros do you want us to keep this numerically backward compatible and issue a warning that the default will change in a future release? I believe the argument is that the "new" default is generally better, but I'm not quite sure it passes the requirement of a "bug fix allowing a break" at first glance.
adammj commentedon Oct 11, 2024
@tylerjereddy I follow your logic. I wouldn't know what the threshold is to "break" now versus later. But I would suspect most would agree that setting
floating_mean=Trueis better in the long run--especially for most "naive" users, that wouldn't even be aware of the downsides to the previous method.tylerjereddy commentedon Oct 11, 2024
Not sure what is going on, but comments from Jake and others have me concerned so just opening this to make sure the matter is thoroughly considered before cutting a release.
jakevdp commentedon Oct 11, 2024
Pasting my comment from #21277 (comment) since it's relevant I think:
adammj commentedon Oct 11, 2024
(I'll do the same, since I think it's probably worthwhile until we nail down where a single discussion is taking place)
jakevdp commentedon Oct 11, 2024
And here's my reply in the other issue:
I propose we continue the discussion here if there's more to say.
adammj commentedon Oct 11, 2024
@jakevdp Yes, I agree with you on both parts. 1) that it (
floating_mean=True) is the more sensible default and 2) that it should be announced.I guess the question is, is this done in the scipy 1.15 release notes, or some (unknown) future scipy release notes?
dhomeier commentedon Oct 11, 2024
Yes, I was in fact mentioning
fit_meanmainly trying to understand what model astropy has been using so far, and seeing that it currently isTrueby default in the mainastropy.lombscargleand its other implementations, I was close to proposing that the scipy wrapper should also adopt this as new default, say as of Astropy 7.0. But that would create even more confusion in the form of different behaviour as long as we support Scipy <= 1.14.So having
floating_meandefault toFalsewould make our implementation a bit easier as we don't have to check for scipy.version to add a new argument, if needed, but now that we now about it, we can probably live with both.neutrinoceros commentedon Oct 11, 2024
This would be the ideal solution as far as I can tell. I totally get the argument that changing the default is desirable, but making the change progressive over the course of a deprecation period would be much easier on us downstream than the brutal change we're seeing now.
Thanks a lot for suggesting it first !
adammj commentedon Oct 11, 2024
But, presumably, even if the
floating_meandefault change was delayed, you'd have to check for the version eventually. Right?I'm just trying to understand what would be different between breaking it now versus in the future. I think astropy would still need to check the scipy version anyways.
tylerjereddy commentedon Oct 11, 2024
I think that's the whole point of deprecation warnings or future warnings -- just to give the ecosystem some time to adapt; there may also be projects with less developer eyes and cross-project testing than
astropy, so we tend to be conservative this close to the base of the ecosystem.Folks can then start to gradually adjust to avoid the warning, and report any surprises to us as they do so, for a more gradual rollout. Otherwise,
astropyand/or us may end up cutting more releases due to surprises, etc.jakevdp commentedon Oct 11, 2024
I think there's a good argument here for not changing the default at all. The estimator produces a different statistic depending on the value of
fit_mean, sometimes it's the better choice (e.g. with a signal that has an unknown mean) and sometimes it isn't (e.g. the window function signal I mentioned above). No single setting is correct in all situations, and given that I'd lean toward not changing the behavior in order to avoid unnecessary churn downstream. If we were introducing a new algorithm, we might choose a different default. But since the current behavior has existed for probably 20+ years, there's a high bar for changing it, and I'm not convinced the change is warranted.tylerjereddy commentedon Oct 11, 2024
Ok, that's helpful feedback, I think the needle is moving farther away from justifying a breaking change at any point then. Perhaps the docs can clarify some of the subtleties/tradeoffs of option/algorithm choice if they don't already.
adammj commentedon Oct 12, 2024
That's a good point on the docs. More than just the parameter default would have to change, as the docstring is written in several places assuming it's
True.However, I'm unsure about how much—or even what—to add to the Notes section (and how long the maintainers are comfortable with it being). Especially since we all agree the
Truevalue (in general) is the preferred option (makes me wonder if adding that note to the Parameters section is appropriate).neutrinoceros commentedon Oct 12, 2024
for what it's worth, I'm +1 on not changing the default behavior at all if it can be avoided.
I'll still note that the following patch should make astropy more robust to an hypothetical future change:
astropy/astropy#17163
but of course that doesn't accomplish anything for every other user of this function, so it shouldn't weight too much on the decision to change the default or not.
tylerjereddy commentedon Oct 13, 2024
Numerical backwards compatibility was restored with default args in gh-21697, so closing.