Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tempfile.TemporaryDirectory() context manager can fail to propagate exceptions generated within its context #85034

Closed
granchester mannequin opened this issue Jun 3, 2020 · 3 comments
Labels
3.7 (EOL) end of life stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@granchester
Copy link
Mannequin

granchester mannequin commented Jun 3, 2020

BPO 40857
Nosy @serhiy-storchaka, @remilapeyre, @iritkatriel

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-11-25.23:12:30.020>
created_at = <Date 2020-06-03.22:08:19.513>
labels = ['3.7', 'type-bug', 'library']
title = 'tempfile.TemporaryDirectory() context manager can fail to propagate exceptions generated within its context'
updated_at = <Date 2020-11-25.23:12:30.018>
user = 'https://bugs.python.org/granchester'

bugs.python.org fields:

activity = <Date 2020-11-25.23:12:30.018>
actor = 'iritkatriel'
assignee = 'none'
closed = True
closed_date = <Date 2020-11-25.23:12:30.020>
closer = 'iritkatriel'
components = ['Library (Lib)']
creation = <Date 2020-06-03.22:08:19.513>
creator = 'granchester'
dependencies = []
files = []
hgrepos = []
issue_num = 40857
keywords = []
message_count = 3.0
messages = ['370689', '370709', '381868']
nosy_count = 4.0
nosy_names = ['serhiy.storchaka', 'remi.lapeyre', 'iritkatriel', 'granchester']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue40857'
versions = ['Python 3.6', 'Python 3.7']

@granchester
Copy link
Mannequin Author

granchester mannequin commented Jun 3, 2020

When an exception occurs within a tempfile.TemporaryDirectory() context
and the directory cleanup fails, the _cleanup exception_ is propagated,
not the original one. This effectively 'masks' the original exception,
and makes it impossible to catch using a simple 'try'/'except' construct.

----------------------------------------------------------------------------
Code like this:

  import tempfile
  import os
  import sys

try:
with tempfile.TemporaryDirectory() as tempdir:
print(tempdir)
# some code happens here

  except ArithmeticError as exc:
      print('An arithmetic error occurred: {}'.format(exc))

  print('Continuing...')

is effective at catching any ArithmeticError which occurs in the
code fragment but is not otherwise handled. However if, in addition,
an error occues in cleaning up the temporary directory, the exception
which occurred in the code is replaced by the cleanup exception, and is
not be propagated to be caught by the 'except' clause.

For example:

  import tempfile
  import os
  import sys

try:
with tempfile.TemporaryDirectory() as tempdir:
print(tempdir)
n = 1 / 0

  except ArithmeticError as exc:
      print('An arithmetic error occurred: {}'.format(exc))

  print('Continuing...')

produces this:

/tmp/tmp_r2sxqgb
An arithmetic error occurred: division by zero
Continuing...

but this:

  import tempfile
  import os
  import sys

try:
with tempfile.TemporaryDirectory() as tempdir:
print(tempdir)
os.rmdir(tempdir) # this new line is the only difference
n = 1 / 0

  except ArithmeticError as exc:
      print('An arithmetic error occurred: {}'.format(exc))

  print('Continuing...')

produces this:

  /tmp/tmp_yz6zyfs
  Traceback (most recent call last):
    File "tempfilebug.py", line 9, in <module>
      n = 1 / 0
  ZeroDivisionError: division by zero

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "tempfilebug.py", line 9, in <module>
      n = 1 / 0
    File "/usr/lib/python3.6/tempfile.py", line 948, in __exit__
      self.cleanup()
    File "/usr/lib/python3.6/tempfile.py", line 952, in cleanup
      _rmtree(self.name)
    File "/usr/lib/python3.6/shutil.py", line 477, in rmtree
      onerror(os.lstat, path, sys.exc_info())
    File "/usr/lib/python3.6/shutil.py", line 475, in rmtree
      orig_st = os.lstat(path)
  FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp_yz6zyfs'

and the program exits with the top-level code having no chance to catch
the ZeroDivisionError and continue execution. (To catch this exception,
the top-level code would need to know to catch FileNotFoundError.)

My view is that if an exception happens within a TemporaryDirectory
context, *and* there is an exception generated as a result of the cleanup
process, the original exception is likely to be more significant, and
should be the exception which is propagated, not the one generated by
the cleanup.

----------------------------------------------------------------------------
System info:

$ python3 --version
Python 3.6.9

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.4 LTS
Release:        18.04
Codename:       bionic

@granchester granchester mannequin added extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error labels Jun 3, 2020
@remilapeyre
Copy link
Mannequin

remilapeyre mannequin commented Jun 4, 2020

This was fixed in e9b51c0 so it's fixed in Python 3.8 and 3.9. Maybe it should have been backported to 3.7.

Python 3.6 only receives security fixes now.

See also bpo-26660 and bpo-35144.

@remilapeyre remilapeyre mannequin added stdlib Python modules in the Lib dir 3.7 (EOL) end of life and removed extension-modules C modules in the Modules dir labels Jun 4, 2020
@iritkatriel
Copy link
Member

Too late for 3.7 now, closing.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.7 (EOL) end of life stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

1 participant