Skip to content

MAINT: lombscargle numerical backward-compat #21696

Closed

Description

@tylerjereddy

Per discussion in gh-21277, I agree it would be best to switch the new lombscargle implementation to use floating_mean=False by default because of the disruption in downstream numerical results we are causing. If it were truly fixing a bug it would normally be allowed to cause a "break", but seems like it is more like a general improvement we could issue a warning of some sort about (i.e., warn that the "better" setting will eventually become the default if the consumer doesn't explicitly specify a value for it?).

Activity

  1. added this to the 1.15.0 milestone on Oct 11, 2024
  2. tylerjereddy commented on Oct 11, 2024

    @tylerjereddy
    ContributorAuthor

    @mhvk @jakevdp @neutrinoceros do you want us to keep this numerically backward compatible and issue a warning that the default will change in a future release? I believe the argument is that the "new" default is generally better, but I'm not quite sure it passes the requirement of a "bug fix allowing a break" at first glance.

  3. adammj commented on Oct 11, 2024

    @adammj
    Contributor

    @tylerjereddy I follow your logic. I wouldn't know what the threshold is to "break" now versus later. But I would suspect most would agree that setting floating_mean=True is better in the long run--especially for most "naive" users, that wouldn't even be aware of the downsides to the previous method.

  4. tylerjereddy commented on Oct 11, 2024

    @tylerjereddy
    ContributorAuthor

    Not sure what is going on, but comments from Jake and others have me concerned so just opening this to make sure the matter is thoroughly considered before cutting a release.

  5. jakevdp commented on Oct 11, 2024

    @jakevdp
    Member

    Pasting my comment from #21277 (comment) since it's relevant I think:

    One place where fit_mean=True will cause problems is when you use lombscargle to find the window frequency by passing in a unit signal, like this:

    signal_psd = lombscargle(x, y, freqs)
    window_psd = lombscargle(x, np.ones_like(y), freqs)

    If fit_mean is True by default, then window_psd is under-constrained, and you'd have to explicitly set fit_mean=False to get meaningful output. This is not "backward-compatible" even in the looser sense of the term.

  6. adammj commented on Oct 11, 2024

    @adammj
    Contributor

    (I'll do the same, since I think it's probably worthwhile until we nail down where a single discussion is taking place)

    I'm confused. fit_mean is a astropy.lombscargle parameter, and it is true by default. And, as best as I can tell, it is equivalent (in spirit) to floating_mean in scipy.lombscargle, which (currently) is also true by default. So, it looks like to me, the two methods are in agreement that a "floating mean" is best, and should be the default.

  7. jakevdp commented on Oct 11, 2024

    @jakevdp
    Member

    And here's my reply in the other issue:

    Yes, in astropy it has always been set to True, because it's the more sensible default when the option is available.

    Here the argument is that changing the default is a breaking change, and so it shouldn't be done quietly because it will break downstream projects (as it did in the linked astropy issue).

    I propose we continue the discussion here if there's more to say.

  8. added
    needs-decisionItems that need further discussion before they are merged or closed
    on Oct 11, 2024
  9. adammj commented on Oct 11, 2024

    @adammj
    Contributor

    @jakevdp Yes, I agree with you on both parts. 1) that it (floating_mean=True) is the more sensible default and 2) that it should be announced.

    I guess the question is, is this done in the scipy 1.15 release notes, or some (unknown) future scipy release notes?

  10. dhomeier commented on Oct 11, 2024

    @dhomeier
    Contributor

    Yes, I was in fact mentioning fit_mean mainly trying to understand what model astropy has been using so far, and seeing that it currently is True by default in the main astropy.lombscargle and its other implementations, I was close to proposing that the scipy wrapper should also adopt this as new default, say as of Astropy 7.0. But that would create even more confusion in the form of different behaviour as long as we support Scipy <= 1.14.
    So having floating_mean default to False would make our implementation a bit easier as we don't have to check for scipy.version to add a new argument, if needed, but now that we now about it, we can probably live with both.

  11. neutrinoceros commented on Oct 11, 2024

    @neutrinoceros

    do you want us to keep this numerically backward compatible and issue a warning that the default will change in a future release?

    This would be the ideal solution as far as I can tell. I totally get the argument that changing the default is desirable, but making the change progressive over the course of a deprecation period would be much easier on us downstream than the brutal change we're seeing now.
    Thanks a lot for suggesting it first !

  12. adammj commented on Oct 11, 2024

    @adammj
    Contributor

    So having floating_mean default to False would make our implementation a bit easier as we don't have to check for scipy.version to add a new argument, if needed, but now that we now about it, we can probably live with both.

    But, presumably, even if the floating_mean default change was delayed, you'd have to check for the version eventually. Right?

    I'm just trying to understand what would be different between breaking it now versus in the future. I think astropy would still need to check the scipy version anyways.

  13. tylerjereddy commented on Oct 11, 2024

    @tylerjereddy
    ContributorAuthor

    I'm just trying to understand what would be different between breaking it now versus in the future.

    I think that's the whole point of deprecation warnings or future warnings -- just to give the ecosystem some time to adapt; there may also be projects with less developer eyes and cross-project testing than astropy, so we tend to be conservative this close to the base of the ecosystem.

    Folks can then start to gradually adjust to avoid the warning, and report any surprises to us as they do so, for a more gradual rollout. Otherwise, astropy and/or us may end up cutting more releases due to surprises, etc.

  14. jakevdp commented on Oct 11, 2024

    @jakevdp
    Member

    I think there's a good argument here for not changing the default at all. The estimator produces a different statistic depending on the value of fit_mean, sometimes it's the better choice (e.g. with a signal that has an unknown mean) and sometimes it isn't (e.g. the window function signal I mentioned above). No single setting is correct in all situations, and given that I'd lean toward not changing the behavior in order to avoid unnecessary churn downstream. If we were introducing a new algorithm, we might choose a different default. But since the current behavior has existed for probably 20+ years, there's a high bar for changing it, and I'm not convinced the change is warranted.

  15. tylerjereddy commented on Oct 11, 2024

    @tylerjereddy
    ContributorAuthor

    Ok, that's helpful feedback, I think the needle is moving farther away from justifying a breaking change at any point then. Perhaps the docs can clarify some of the subtleties/tradeoffs of option/algorithm choice if they don't already.

  16. adammj commented on Oct 12, 2024

    @adammj
    Contributor

    That's a good point on the docs. More than just the parameter default would have to change, as the docstring is written in several places assuming it's True.

    However, I'm unsure about how much—or even what—to add to the Notes section (and how long the maintainers are comfortable with it being). Especially since we all agree the True value (in general) is the preferred option (makes me wonder if adding that note to the Parameters section is appropriate).

  17. neutrinoceros commented on Oct 12, 2024

    @neutrinoceros

    for what it's worth, I'm +1 on not changing the default behavior at all if it can be avoided.
    I'll still note that the following patch should make astropy more robust to an hypothetical future change:
    astropy/astropy#17163
    but of course that doesn't accomplish anything for every other user of this function, so it shouldn't weight too much on the decision to change the default or not.

  18. tylerjereddy commented on Oct 13, 2024

    @tylerjereddy
    ContributorAuthor

    Numerical backwards compatibility was restored with default args in gh-21697, so closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    maintenanceItems related to regular maintenance tasksneeds-decisionItems that need further discussion before they are merged or closedscipy.signal

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions