register default deprecation warning filter. #8480

Carreau · 2015-05-27T17:57:30Z

warn user if (s)he use a deprecated feature interactively.

In [1]: %run -i foo
In [2]: foo()
foo.py:4: DeprecationWarning: using a non-integer number instead of an
    integer will result in an error in the future
    print('x'*np.float64(3.5))
xxx

In [3]: from foo import foo
In [4]: foo()
xxx

Not sure it is the exact right place though.

warn user if (s)he use a deprecated feature interactively. Closes ipython#8478 In [1]: %run -i foo In [2]: foo() foo.py:4: DeprecationWarning: using a non-integer number instead of an integer will result in an error in the future print('x'*np.float64(3.5)) xxx In [3]: from foo import foo In [4]: foo() xxx

Carreau · 2015-05-27T18:29:53Z

Test added.

Would @njsmith like to try ? And would that suit him.

njsmith · 2015-05-27T19:46:07Z

Looks reasonable to me.

It is probably not quite right in some more obscure corner cases, but I'm not sure how much we should care about that. Specifically I am thinking of code run non-interactively through the InteractiveShell (does it exist? run_last_nodes takes an interactivity argument...), and InteractiveShells that are not using the main namespace. Do we care? (This is why I didn't submit a PR myself :-).)

minrk · 2015-05-27T20:17:38Z

It is possible for code to be run non-interactively with InteractiveShell, but I think it is appropriate for any use of InteractiveShell to behave as if interactively executed.

👍 to this

Carreau · 2015-05-27T20:18:31Z

Yes you can run things with execute preprocessor, but then you still want stderr to be captured in the notebook (for the notebook case). So +1 on behaving the same. We can refine the behavior.

I should add a note to the what's new.

Carreau · 2015-05-27T20:21:50Z

Hum, importing a deprecated module in main does not print the warning either...

Carreau · 2015-05-27T20:35:44Z

Side question, why in #6680 do we go for clearing the registry instead of "always" filters ?

njsmith · 2015-05-27T20:38:52Z

we don't have total control over the warning filter; the hack there
makes the existing filter work right.

for i in range(100):
do_something_deprecated()
^^ with "default" filters and #6680 this will emit one warning each time
the cell is executed. With "always" it will emit 100 warnings every time
the cell is executed.
On May 27, 2015 1:36 PM, "Matthias Bussonnier" [email protected]
wrote:

Side question, why in #6680 #6680
do we go for clearing the registry instead of "always" filters ?

—
Reply to this email directly or view it on GitHub
#8480 (comment).

Carreau · 2015-05-27T20:49:57Z

Ah, make sens !

njsmith · 2015-05-27T23:16:47Z

Hum, importing a deprecated module in main does not print the warning either...

Ah, this appears to be because the import system rewrite in CPython has totally screwed up module deprecation.

A key feature of warnings is that they are attributed -- they are supposed to contain a pointer to the code that needs fixing. This is done via the stacklevel argument to warnings.warn -- the default warnings.warn("foo", stacklevel=1) attributes the warning to the line of code that called warnings.warn, which is basically always wrong; or you can use a higher value of stacklevel= to attribute the warning to your caller, or your caller's caller, etc. E.g., here's the correct way to warn for a deprecated function:

def foo():
    warnings.warn("foo() is deprecated", DeprecationWarning, stacklevel=2)

foo()

Here the warning will be attributed to the last line of code, not to the body of foo.

This is crucial for our warnings filter -- we're requesting that we only see warnings that are attributed to code running in the __main__ module.

So if you call warnings.warn at the top-level inside a module, then what happens? Who is the 'caller' of the body of a module? Let's find out:

# w.py
import sys
import warnings
import traceback

sys.stdout.write("Nominal call stack:\n")
traceback.print_stack()

sys.stdout.write("\nWarnings:\n")
for i in range(1, 12):
    warnings.warn("At import, stacklevel %s" % (i,), stacklevel=i)
    sys.stdout.write("\n")

Output on python 2:

>>> import w
Nominal call stack:
  File "<stdin>", line 1, in <module>
  File "w.py", line 6, in <module>
    traceback.print_stack()

Warnings:
w.py:10: UserWarning: At import, stacklevel 1
  warnings.warn("At import, stacklevel %s" % (i,), stacklevel=i)

__main__:1: UserWarning: At import, stacklevel 2

sys:1: UserWarning: At import, stacklevel 3

[...trimmed...]

so stacklevel=2 does the right thing, as you'd expect.

On python 3.3:

>>> import w
Nominal call stack:
  File "<stdin>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 1565, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1532, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 584, in _check_name_wrapper
  File "<frozen importlib._bootstrap>", line 1022, in load_module
  File "<frozen importlib._bootstrap>", line 1003, in load_module
  File "<frozen importlib._bootstrap>", line 560, in module_for_loader_wrapper
  File "<frozen importlib._bootstrap>", line 868, in _load_module
  File "<frozen importlib._bootstrap>", line 313, in _call_with_frames_removed
  File "./w.py", line 6, in <module>
    traceback.print_stack()

Warnings:
./w.py:10: UserWarning: At import, stacklevel 1
  warnings.warn("At import, stacklevel %s" % (i,), stacklevel=i)

/usr/lib/python3.3/importlib/_bootstrap.py:313: UserWarning: At import, stacklevel 2
  return f(*args, **kwds)

/usr/lib/python3.3/importlib/_bootstrap.py:868: UserWarning: At import, stacklevel 3
  _call_with_frames_removed(exec, code_object, module.__dict__)

/usr/lib/python3.3/importlib/_bootstrap.py:560: UserWarning: At import, stacklevel 4
  return fxn(self, module, *args, **kwargs)

/usr/lib/python3.3/importlib/_bootstrap.py:1003: UserWarning: At import, stacklevel 5
  return self._load_module(fullname)

/usr/lib/python3.3/importlib/_bootstrap.py:1022: UserWarning: At import, stacklevel 6
  return super(FileLoader, self).load_module(fullname)

/usr/lib/python3.3/importlib/_bootstrap.py:584: UserWarning: At import, stacklevel 7
  return method(self, name, *args, **kwargs)

/usr/lib/python3.3/importlib/_bootstrap.py:1532: UserWarning: At import, stacklevel 8
  loader.load_module(name)

/usr/lib/python3.3/importlib/_bootstrap.py:1565: UserWarning: At import, stacklevel 9
  return _find_and_load_unlocked(name, import_)

__main__:1: UserWarning: At import, stacklevel 10

sys:1: UserWarning: At import, stacklevel 11

So on python 3.3, if you want to correctly deprecate a module, you have to write warnings.warn("this module is deprecated", DeprecationWarning, stacklevel=10). Duh.

And on python 3.4, it changed again:

>>> import w
Nominal call stack:
  File "<stdin>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "<frozen importlib._bootstrap>", line 1471, in exec_module
  File "<frozen importlib._bootstrap>", line 321, in _call_with_frames_removed
  File "/tmp/w.py", line 6, in <module>
    traceback.print_stack()

Warnings:
/tmp/w.py:10: UserWarning: At import, stacklevel 1
  warnings.warn("At import, stacklevel %s" % (i,), stacklevel=i)

/usr/lib/python3.4/importlib/_bootstrap.py:321: UserWarning: At import, stacklevel 2
  return f(*args, **kwds)

/usr/lib/python3.4/importlib/_bootstrap.py:1471: UserWarning: At import, stacklevel 3
  _call_with_frames_removed(exec, code, module.__dict__)

/usr/lib/python3.4/importlib/_bootstrap.py:1129: UserWarning: At import, stacklevel 4
  self.spec.loader.exec_module(module)

/usr/lib/python3.4/importlib/_bootstrap.py:1200: UserWarning: At import, stacklevel 5
  self._exec(module)

/usr/lib/python3.4/importlib/_bootstrap.py:2226: UserWarning: At import, stacklevel 6
  module = _SpecMethods(spec)._load_unlocked()

/usr/lib/python3.4/importlib/_bootstrap.py:2237: UserWarning: At import, stacklevel 7
  return _find_and_load_unlocked(name, import_)

__main__:1: UserWarning: At import, stacklevel 8

sys:1: UserWarning: At import, stacklevel 9

sys:1: UserWarning: At import, stacklevel 10

sys:1: UserWarning: At import, stacklevel 11

So basically the conclusion is that it's not possible to correctly deprecate a module on Python 3. Sweet!

njsmith · 2015-05-27T23:37:53Z

Filed a bug upstream: https://bugs.python.org/issue24305

njsmith · 2015-05-27T23:39:11Z

IPython/core/interactiveshell.py

Maybe this should instead say

module=self.user_ns.get("__name__")

?

njsmith · 2015-05-27T23:39:47Z

Anyway, chatter aside, this LGTM except for the minor comment about using user_ns["__name__"].

Carreau · 2015-05-28T02:12:28Z

Anyway, chatter aside, this LGTM except for the minor comment about using user_ns["name"].

done.

Carreau · 2015-05-28T06:43:30Z

Waiting on CPython to choose a behavior.
We might deactivate this for PendingDW IIUC.

See mail thread https://mail.python.org/pipermail/python-ideas/2015-May/033740.html and bug on CPython http://bugs.python.org/issue24294

minrk · 2015-05-28T18:33:10Z

I'm 50/50 on whether we unhide PendingDeprecation. Based on Nick's comment, I might be slightly inclined to continue ignoring them, and only unhide actual deprecations.

Carreau · 2015-05-28T19:10:47Z

Only show deprecation warnings now.

register default deprecation warning filter.

…o the user. Generating deprecation warnings with the correct stacklevel is tricky to get right. Up to now, users have not been seeing these deprecation warnigns because they had incorrect stacklevels. See python/cpython#68493 and python/cpython#67998 for good discussions, as well as ipython/ipython#8478 (comment) and ipython/ipython#8480 (comment) for more context. We see these issues directly. Normally stacklevel=2 might be enough to target the caller of a function. However, in our deprecation warning in the __init__ method, each level of subclasses adds one stack frame, so depending on the subclass, the appropriate stacklevel is different. Thus this PR implements a trick from a discussion in Python to calculate the first stack frame that is external to the ipywidgets library, and use that as the target stack frame for the deprecation warning.

Carreau added 2 commits May 27, 2015 10:53

Add test that interactive warning is shown.

a498028

Carreau force-pushed the interactive-deprecation-warning branch from 83b2cd8 to a498028 Compare May 27, 2015 18:09

Test imported code does not raise deprecation warning.

4763214

add info in whats new

c69d5c4

njsmith reviewed May 27, 2015
View reviewed changes

IPython/core/interactiveshell.py Outdated

Copy link

Contributor

njsmith May 27, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this should instead say

module=self.user_ns.get("__name__")

?

njsmith mentioned this pull request May 28, 2015

DeprecationWarnings should be visible by default in interactive usage #8478

Closed

use user-ns __name__

fb7adcb

do not show pending deprecation warnign

e1b98ee

minrk added a commit that referenced this pull request Jun 1, 2015

Merge pull request #8480 from Carreau/interactive-deprecation-warning

3a35a5e

register default deprecation warning filter.

minrk merged commit 3a35a5e into ipython:master Jun 1, 2015

minrk modified the milestone: 4.0 Aug 11, 2015

Carreau deleted the interactive-deprecation-warning branch May 23, 2016 17:47

Carreau mentioned this pull request Sep 27, 2018

Remove custom rule for deprecation warnigns in __main__ #11333

Open

jasongrout mentioned this pull request Aug 26, 2022

Calculate correct stack level for deprecation warnings to show them to the user jupyter-widgets/ipywidgets#3569

Merged

Uh oh!

register default deprecation warning filter. #8480

register default deprecation warning filter. #8480

Uh oh!

Conversation

Carreau commented May 27, 2015

Uh oh!

Carreau commented May 27, 2015

Uh oh!

njsmith commented May 27, 2015

Uh oh!

minrk commented May 27, 2015

Uh oh!

Carreau commented May 27, 2015

Uh oh!

Carreau commented May 27, 2015

Uh oh!

Carreau commented May 27, 2015

Uh oh!

njsmith commented May 27, 2015

Uh oh!

Carreau commented May 27, 2015

Uh oh!

njsmith commented May 27, 2015

Uh oh!

njsmith commented May 27, 2015

Uh oh!

njsmith May 27, 2015

Choose a reason for hiding this comment

Uh oh!

njsmith commented May 27, 2015

Uh oh!

Carreau commented May 28, 2015

Uh oh!

Carreau commented May 28, 2015

Uh oh!

minrk commented May 28, 2015

Uh oh!

Carreau commented May 28, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants