Skip to content

Document that with pytest.warns() captures all warnings, not just those it tests were emitted #9288

Closed

Description

@Kojoley

I have a test that ensures a specific warning type is produced by a function, but since I switched to pytest.warns I no longer see Python 3.10 compatibility warnings in Pytest warnings summary the code was emitting before switching to pytest.warns. Is it expected behavior (in which case it contradicts https://docs.pytest.org/en/6.2.x/warnings.html#deprecationwarning-and-pendingdeprecationwarning) and how I can opt-out of hiding DeprecationWarnings by pytest.warns (there is nothing about that in the documentation https://docs.pytest.org/en/6.2.x/reference.html#pytest-warns)?

MVCE:

import pytest
import warnings

def test_warning():
    with pytest.warns(DeprecationWarning):
        with pytest.warns(UserWarning):  # catches UserWarning, but also every other type too
            warnings.warn("my warning", UserWarning)
            warnings.warn("some deprecation warning", DeprecationWarning)
================================== FAILURES ===================================
________________________________ test_warning _________________________________

    def test_warning():
>       with pytest.warns(DeprecationWarning):
E       Failed: DID NOT WARN. No warnings of type (<class 'DeprecationWarning'>,) was emitted. The list of emitted warnings is: [].

Activity

  1. changed the title [-]`pytest.warns` hides non-expected warnings, even `DeprecationWarning`[/-] [+]`pytest.warns` hides not expected warnings, even `DeprecationWarning`[/+] on Nov 9, 2021
  2. added
    plugin: warningsrelated to the warnings builtin plugin
    type: questiongeneral question, might be closed after 2 weeks of inactivity
    on Nov 10, 2021
  3. nicoddemus commented on Nov 10, 2021

    @nicoddemus
    Member

    Hi @Kojoley,

    This is the expected behavior, pytest.warns acts as a net which captures all warnings inside the block, ensuring the given warning is raised, or failing otherwise.

    We might add a note to https://docs.pytest.org/en/6.2.x/warnings.html#deprecationwarning-and-pendingdeprecationwarning to mention that behavior does not apply for code inside pytest.warns, what do you think?

  4. Kojoley commented on Nov 10, 2021

    @Kojoley
    Author

    The pytest.warns documentation does not say it will filter everything (actually it does not say it will filter anything at all), so it should be fixed too. It preferably should say how to make pytest.warns to not capture everything or an alternative solution, but I still myself have no idea how to achieve that with tools Pytest provides, could you please suggest anything?

  5. nicoddemus commented on Nov 10, 2021

    @nicoddemus
    Member

    The pytest.warns documentation does not say it will filter everything (actually it does not say it will filter anything at all), so it should be fixed too.

    True, thanks!

    I still myself have no idea how to achieve that with tools Pytest provides, could you please suggest anything?

    Hmm I can't think of anything right now, pytest.warns uses warnings.catch_warnings under the covers, which by design captures all warnings.

  6. nicoddemus commented on Nov 10, 2021

    @nicoddemus
    Member

    Created #9291 to track the docs change.

  7. Kojoley commented on Nov 10, 2021

    @Kojoley
    Author

    If I understand correctly pytest.deprecated_call hides all warnings by design too?

    Also, https://docs.pytest.org/en/6.2.x/warnings.html#warns says You can check that code raises a particular warning using func:pytest.warns, which works in a similar manner to raises: but raises does not hide all exceptions, example:

    def test_raises():
        with pytest.raises(RuntimeError):              # passes,
            with pytest.raises(NotImplementedError):   # because this will not catch and hide the exception
                raise RuntimeError("some exception")
  8. nicoddemus commented on Nov 10, 2021

    @nicoddemus
    Member

    If I understand correctly pytest.deprecated_call hides all warnings by design too?

    Yes. 👍

    but raises does not hide all exceptions, example

    Good point, we need to mention that difference in the docs too.

  9. added
    type: docsdocumentation improvement, missing or needing clarification
    and removed
    type: questiongeneral question, might be closed after 2 weeks of inactivity
    on Nov 29, 2021
  10. changed the title [-]`pytest.warns` hides not expected warnings, even `DeprecationWarning`[/-] [+]Document that `with pytest.warns()` captures all warnings, not just those it tests were emitted[/+] on Apr 21, 2022
  11. Zac-HD commented on Apr 21, 2022

    @Zac-HD
    Member

    Alternatively we could make pytest.warns() re-emit all captured warnings on exit, which might better match users' intuitions - including mine, from before I went and looked up warnings.catch_warnings(record=True).

  12. QuLogic commented on Jan 13, 2023

    @QuLogic

    I think it would make sense to re-raise all unmatched warnings. For example, say you are testing your own deprecated code so you add pytest.warns(DeprecationWarning, match='something specific to your project'). Then one of your dependencies (or maybe some other part of your code) adds a deprecation that is only triggered by that original code's tests. Then you will be blind-sided if the second deprecation is removed first because you never saw it.

  13. zanieb commented on Apr 4, 2023

    @zanieb

    Note that this also means that if you pass an incorrect value to warnings.warn it will hide the error. See #10865

  14. Zac-HD commented on Jul 3, 2023

    @Zac-HD
    Member

    As of #10937, expected in Pytest 8.0, we re-emit all non-matching warnings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    plugin: warningsrelated to the warnings builtin plugintype: docsdocumentation improvement, missing or needing clarification

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions