Python Adventures

Problem

In a project of mine I had to deal with folders, where a folder can contain several thousands of small text files. I kept this project on Dropbox, so I could use it on all my machines. However, Dropbox is quite slow when trying to synchronize several thousand files. So I decided to put files in a folder into a zip file.

So the question is: how to deal with zip files? How to do basic operations with them: create zip, delete from zip, list zip, add to zip, move to zip, extract from zip, etc.

Solution

In this project of mine I used the external zip command as well as the zipfile package from the stdlib. Let’s see both of them.

Manipulating zip files from the command-line
Let’s see some examples. Compress every .json file in the current directory except the desc.json file:

zip -9 files.zip *.json -x desc.json

The switch “-9” gives the best compression, files.zip is the output, and “-x” is short for “--exclude“. From Python you can call it as an external command with os.system() for instance.

The previous example creates a zip file and leaves the original files. Now let’s move files into a zip file (and delete the original files when they were added successfully to the archive):

zip -9 -m files.zip *.json -x desc.json

Delete a file from an archive:

zip -d files.zip desc.json

It will delete desc.json from the zip file.

List the content of a zip file:

zipinfo files.zip

Add a file to the archive:

zip -g files.zip new.json

Where “-g” means: grow.

Extract just one file from a zip file:

# basic:
unzip files.zip this.json

# extract to a specific folder:
unzip files.zip this.json -d /extract/here/

It will extract this.json from the archive.

Read the content of a zip file in Python
OK, say we have a zip file that contains some files. How to get the filenames? How to read them? I found some nice examples here.

List the file names in a zip file:

import zipfile

zfile = zipfile.ZipFile("files.zip", "r")

for name in zfile.namelist():
    print(name)

Read files in a zip file:

import zipfile

zfile = zipfile.ZipFile("files.zip", "r")

for name in zfile.namelist():
    data = zfile.read(name)
    print(data)

Links

• The zipfile module at effbot.org.