Archive
ChDir: a context manager for switching working directories
Problem
In your program you want to change the working directory temporarily, do some job there, then switch back to the original directory. Say you want to download some images to /tmp. When done, you want to get back to the original location correctly, even if an exception was raised at the temp. location.
Naïve way
Let’s see the following example. We have a script, say at /home/jabba/python/fetcher.py . We want to download some images to /tmp, then work with them. After the download we want to create a subfolder “process” in the same directory where the script fetcher.py is located. We want to collect some extra info about the downloaded images and we want to store these pieces of information in the “process” folder.
import os
def download(li, folder):
try:
backup = os.getcwd()
os.chdir(folder)
for img in li:
# download img somehow
os.chdir(backup)
except:
# problem with download, handle it
def main():
# step 1: download images to /tmp
li = ["http://...1.jpg", "http://...2.jpg", "http://...3.jpg"]
download(li, "/tmp")
# step 2: create a "process" dir. HERE (where the script was launched)
os.mkdir("process")
# ...do some extra work...
There is a problem with the download method. If an image cannot be downloaded correctly and an exception occurs, we return from the method. However, os.chdir(backup) is not executed and we remain in the /tmp folder! In main() in step 2 the process directory will be created in /tmp and not in the folder where we wanted it to be.
Well, you can always add a finally block to the exception handler and place os.chdir(backup) there, but it’s easy to forget. Is there an easier solution?
Solution
Yes, there is an easier solution. Use a context manager.
The previous example with a context manager:
import os
def download(li, folder):
with ChDir(folder):
for img in li:
# download img somehow
def main():
# step 1: download images to /tmp
li = ["http://...1.jpg", "http://...2.jpg", "http://...3.jpg"]
download(li, "/tmp")
# step 2: create a "process" dir. HERE (where the script was launched)
os.mkdir("process")
# ...do some extra work...
And now the source code of ChDir:
import os
class ChDir(object):
"""
Step into a directory temporarily.
"""
def __init__(self, path):
self.old_dir = os.getcwd()
self.new_dir = path
def __enter__(self):
os.chdir(self.new_dir)
def __exit__(self, *args):
os.chdir(self.old_dir)
Since ChDir is a context manager, you use it in a with block. At the beginning of the block you enter the given folder. When you leave the with block (even if you leave because of an exception), you are put back to the folder where you were before entering the with block.
Update
Following this discussion thread @reddit, someone suggested using the PyFilesytem library. I think PyFilesytem is a very good solution but it may be too much for a short script. It’s like shooting a sparrow with a cannon :) For a simple script ChDir is good enough for me. For a serious application, check out PyFilesytem.
How to ignore an exception — the elegant way
This idea was presented by Raymond Hettinger at PyCon US 2013. He is talking about it at 43:30: http://www.youtube.com/watch?v=OSGv2VnC0go.
Problem
You want to ignore an exception.
Solution 1: the classical way
Say you want to delete a file but it’s not sure it exists.
try:
os.unlink('somefile.txt')
except OSError:
pass
That is, if the exception occurs, we do nothing.
Solution 2: the elegant way
with ignored(OSError):
os.unlink('somefile.txt')
Its source code in Python 2.7:
from contextlib import contextmanager
@contextmanager
def ignored(*exceptions):
try:
yield
except exceptions:
pass
This is part of Python 3.4, thus in Python 3.4 all you need is this line:
from contextlib import ignored
See the docs here.
Autoflush
Printing to the standard output is buffered. What to do if you want to see the output immediately?
import sys import os # reopen stdout file descriptor with write mode # and 0 as the buffer size (unbuffered) sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0) print "unbuffered text"
Credits
Update (20130206)
The solution above switches buffered mode off, but you can’t switch it back on because you lose the original sys.stdout file descriptor. I have a more sophisticated solution, available here (autoflush.py) as part of my jabbapylib library.
Usage #1:
autoflush(True) # text that you want to print in unbuffered mode comes here autoflush(False) # back to normal
Usage #2:
# using a context manager
with AutoFlush():
# unbuffered text comes here
sys.stdout.write(...)
# here you are back to normal
Let’s not forget the simplest and most trivial solution either:
sys.stdout.write(...) sys.stdout.flush() # flush out immediately
Update (20210827)
Here is a Python 3 solution:
# auto-flush
sys.stdout = io.TextIOWrapper(
open(sys.stdout.fileno(), 'wb', 0),
write_through=True
)
Python Adventures 