Repository navigation
Raised exception in Enum keeping user objects alive unnecessarily #86414
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/ethanfurman'
closed_at = <Date 2021-11-13.19:10:16.809>
created_at = <Date 2020-11-02.20:46:49.158>
labels = ['3.8', 'type-bug', 'library', '3.9', '3.10']
title = 'Raised exception in Enum keeping user objects alive unnecessarily'
updated_at = <Date 2021-11-13.19:10:16.809>
user = 'https://github.com/efroemling'bugs.python.org fields:
activity = <Date 2021-11-13.19:10:16.809>
actor = 'ethan.furman'
assignee = 'ethan.furman'
closed = True
closed_date = <Date 2021-11-13.19:10:16.809>
closer = 'ethan.furman'
components = ['Library (Lib)']
creation = <Date 2020-11-02.20:46:49.158>
creator = 'efroemling'
dependencies = []
files = ['49564']
hgrepos = []
issue_num = 42248
keywords = ['patch']
message_count = 12.0
messages = ['380249', '382523', '390093', '390095', '390859', '390889', '390913', '391285', '403860', '403861', '403862', '403875']
nosy_count = 5.0
nosy_names = ['mark.dickinson', 'pitrou', 'ethan.furman', 'efroemling', 'gerald.dalley2']
pr_nums = ['25350', '25369', '25370']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue42248'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']
Activity
efroemling commentedon Nov 2, 2020
I've run into an issue where exceptions thrown by Enum constructors are keeping my objects alive. The underlying issue seems to be the same as https://bugs.python.org/issue36820
The same method used to fix the issue above seems to work here: simply adding a try/finally clause around the error handling at the end of enum.Enum.__new__() which sets ve_exc and exc to None does the trick.
I've attached a short script which demonstrates the issue. I realize that the cyclic garbage collector will eventually handle this case, but its a bummer to lose determinism in the destruction of my objects.
I'd be happy to create a PR for this or whatever I can do to help; just let me know if I should (I'm new here).
geralddalley2 commentedon Dec 4, 2020
I and a few others have run into issues with the Enum constructors producing spurious reference cycles. This can cause memory explosions if large objects like numpy arrays are held in any of the relevant stack frames. Based on https://bugs.python.org/issue36820, it looks like the maintainers of CPython are open to fixing similar issues, and PRs look like the way to make progress.
mdickinson commentedon Apr 2, 2021
I'm running into this, too, on Python 3.9.2; in my case it's causing segmentation faults in a wxPython-based application.
Those segfaults aren't the fault of the reference cycle, of course; they're a result of wxPython being finicky about object cleanup order, but the existence of the enum reference cycle does make it harder to maneuver wxPython into a place where it doesn't crash.
mdickinson commentedon Apr 2, 2021
Here's a cut-down example, at the prompt:
At this point, since the App() instance was local to create_app, and wasn't returned, I'd expect there to be no App objects alive in the system. But it turns out there's still an App object being kept alive:
The cause is a call to _int_to_enum in signal.py which attempts to coerce the default signal handler to an element of Handlers. That coercion fails, leaving an exception
The traceback on that exception leads to the frame containing the call to Enum.__new__, which in turn contains a reference ve_exc back to the exception.
[In the real code, App was a wx.App object, and the App.__init__ method played the role of create_app.]
ethanfurman commentedon Apr 12, 2021
New changeset 8c14f5a by Ethan Furman in branch 'master':
bpo-42248: [Enum] ensure exceptions raised in
_missing_are released (GH-25350)8c14f5a
ethanfurman commentedon Apr 12, 2021
New changeset f396a1a by Ethan Furman in branch '3.8':
[3.8] bpo-42248: [Enum] ensure exceptions raised in
_missing_are released (GH-25350). (GH-25369)f396a1a
ethanfurman commentedon Apr 12, 2021
New changeset 6379924 by Ethan Furman in branch '3.9':
[3.9] bpo-42248: [Enum] ensure exceptions raised in
_missing_are released (GH-25350). (GH-25370)6379924
5 remaining items
mdickinson commentedon Apr 17, 2021
Thank you for the quick fix!
pitrou commentedon Oct 13, 2021
It appears this is *not* fixed in 3.10.0:
pitrou commentedon Oct 13, 2021
Indeed, changeset 8c14f5a is part of git main, but is not on the 3.10 branch AFAICT.
pitrou commentedon Oct 13, 2021
It seems the problem is that bpo-44559 reset the enum module to a previous state without the bugfix.
ethanfurman commentedon Oct 13, 2021
Going back through the various bug fixes that got cut from 3.10 to re-add to 3.10.1.