Dive Into Python 3

■■■

Mark Pilgrim

i
 Dive Into Python 3
Copyright © 2009 by Mark Pilgrim
All rights reserved. No part of this work may be reproduced or transmitted in any form or by
any means, electronic or mechanical, including photocopying, recording, or by any
information storage or retrieval system, without the prior written permission of the copyright
owner and the publisher.
ISBN-13 (pbk): 978-1-4302-2415-0
ISBN-13 (electronic): 978-1-4302-2416-7
Printed and bound in the United States of America 9 8 7 6 5 4 3 2 1
Trademarked names may appear in this book. Rather than use a trademark symbol with every
occurrence of a trademarked name, we use the names only in an editorial fashion and to the
benefit of the trademark owner, with no intention of infringement of the trademark.
Lead Editor: Duncan Parkes
Technical Reviewer: Simon Willison
Editorial Board: Clay Andres, Steve Anglin, Mark Beckner, Ewan Buckingham, Tony
Campbell, Gary Cornell, Jonathan Gennick, Michelle Lowman, Matthew Moodie,
Jeffrey Pepper, Frank Pohlmann, Ben Renow-Clarke, Dominic Shakeshaft, Matt Wade,
Tom Welsh
Project Managers: Richard Dal Porto and Debra Kelly
Copy Editors: Nancy Sixsmith, Heather Lang, Patrick Meader, and Sharon Terdeman
Compositor: folio 2
Indexer: Julie Grady
Distributed to the book trade worldwide by Springer-Verlag New York, Inc., 233 Spring Street,
6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax 201-348-4505, e-mail orders-
ny@[Link], or visit [Link]
For information on translations, please e-mail info@[Link], or visit
[Link]
Apress and friends of ED books may be purchased in bulk for academic, corporate, or
promotional use. eBook versions and licenses are also available for most titles. For more
information, reference our Special Bulk Sales–eBook Licensing web page at
[Link]
The information in this book is distributed on an “as is” basis, without warranty. Although
every precaution has been taken in the preparation of this work, neither the author(s) nor
Apress shall have any liability to any person or entity with respect to any loss or damage
caused or alleged to be caused directly or indirectly by the information contained in this work.

ii
For Michael

iii
 Contents at a Glance

■Foreword........................................................................................................................... xiii
■About the Author ................................................................................................................ xv
■About the Technical Reviewer........................................................................................... xvi
■Acknowledgments............................................................................................................ xvii
■Installing Python ............................................................................................................. xviii
■Chapter 1: Your First Python Program ................................................................................. 1
■Chapter 2: Native Datatypes............................................................................................... 15
■Chapter 3: Comprehensions ............................................................................................... 43
■Chapter 4: Strings .............................................................................................................. 53
■Chapter 5: Regular Expressions ......................................................................................... 69
■Chapter 6: Closures and Generators................................................................................... 87
■Chapter 7: Classes and Iterators ...................................................................................... 101
■Chapter 8: Advanced Iterators ......................................................................................... 113
■Chapter 9: Unit Testing..................................................................................................... 131
■Chapter 10: Refactoring ................................................................................................... 155
■Chapter 11: Files .............................................................................................................. 167
■Chapter 12: XML............................................................................................................... 185
■Chapter 13: Serializing Python Objects ............................................................................ 205
■Chapter 14: HTTP Web Services ....................................................................................... 225
■Chapter 15: Case Study: Porting chardet to Python 3 ...................................................... 253
■Chapter 16: Packaging Python Libraries.......................................................................... 279
■Appendix A: Porting Code to Python 3 with 2to3 ............................................................. 295
■Appendix B: Special Method Names................................................................................. 327
■Appendix C: Where to Go From Here................................................................................. 345
■Index ................................................................................................................................ 347

iv
Contents

■Foreword........................................................................................................................... xiii
■About the Author ................................................................................................................ xv
■About the Technical Reviewer........................................................................................... xvi
■Acknowledgments............................................................................................................ xvii
■Installing Python ............................................................................................................. xviii
Which Python Is Right for You? ...............................................................................................................xi
Installing on Microsoft Windows........................................................................................................... xii
Installing on Mac OS X ...........................................................................................................................xxi
Installing on Ubuntu Linux................................................................................................................xxxiii
Installing on Other Platforms ................................................................................................................ xli
Using the Python Shell ........................................................................................................................... xli
Python Editors and IDEs .......................................................................................................................xliv
■Chapter 1: Your First Python Program ................................................................................. 1
Declaring Functions ..................................................................................................................................2
Optional and Named Arguments..............................................................................................................3
Writing Readable Code..............................................................................................................................5
Documentation Strings .............................................................................................................................5
The import Search Path.............................................................................................................................6
Everything Is an Object .............................................................................................................................7
What’s an Object? ......................................................................................................................................8
Indenting Code ..........................................................................................................................................8
Exceptions ..................................................................................................................................................9
Catching Import Errors ...........................................................................................................................11
Unbound Variables..................................................................................................................................12
Running Scripts .......................................................................................................................................12
Further Reading Online...........................................................................................................................13

v
 ■ CONTENTS

■Chapter 2: Native Datatypes............................................................................................... 15

Booleans ...................................................................................................................................................15
Numbers ...................................................................................................................................................16
Coercing Integers to Floats and Vice Versa ............................................................................................17
Common Numerical Operations .......................................................................................................18
Fractions...................................................................................................................................................19
Trigonometry ...........................................................................................................................................20
Numbers in a Boolean Context ...............................................................................................................20
Lists...........................................................................................................................................................21
Creating a List .....................................................................................................................................21
Slicing a List ........................................................................................................................................22
Adding Items to a List .........................................................................................................................23
Searching For Values in a List ............................................................................................................25
Removing Items from a List ...............................................................................................................26
Removing Items from a List: Bonus Round ......................................................................................26
Lists in a Boolean Context ..................................................................................................................27
Tuples .......................................................................................................................................................28
Tuples in a Boolean Context ..............................................................................................................30
Assigning Multiple Values at Once.....................................................................................................30
Sets............................................................................................................................................................31
Modifying a Set ...................................................................................................................................33
Removing Items from a Set ................................................................................................................34
Common Set Operations ....................................................................................................................35
Sets in a Boolean Context ...................................................................................................................37
Dictionaries..............................................................................................................................................38
Creating a Dictionary .........................................................................................................................38
Modifying a Dictionary.......................................................................................................................38
Mixed-Value Dictionaries...................................................................................................................39
Dictionaries in a Boolean Context .....................................................................................................40
None..........................................................................................................................................................41
None in a Boolean Context.................................................................................................................41
Further Reading Online...........................................................................................................................42
■Chapter 3: Comprehensions ............................................................................................... 43
Working With Files and Directories........................................................................................................43
The Current Working Directory .........................................................................................................43
Working with Filenames and Directory Names ................................................................................44
Listing Directories...............................................................................................................................46
Getting File Metadata .........................................................................................................................47
Constructing Absolute Pathnames ....................................................................................................47
List Comprehensions...............................................................................................................................48
Dictionary Comprehensions...................................................................................................................50

vi
 ■ CONTENTS

Fun with Dictionary Comprehensions ..............................................................................................51

Set Comprehensions ...............................................................................................................................51
Further Reading Online...........................................................................................................................52
■Chapter 4: Strings .............................................................................................................. 53
Unicode ....................................................................................................................................................54
Diving In...................................................................................................................................................56
Formatting Strings...................................................................................................................................56
Compound Field Names..........................................................................................................................57
Format Specifiers.....................................................................................................................................59
Other Common String Methods .............................................................................................................60
Slicing a String .........................................................................................................................................61
Strings versus Bytes .................................................................................................................................62
Character Encoding of Python Source Code..........................................................................................65
Further Reading Online...........................................................................................................................66
■Chapter 5: Regular Expressions ......................................................................................... 69
Case Study: Street Addresses ..................................................................................................................69
Case Study: Roman Numerals ................................................................................................................71
Checking for Thousands .........................................................................................................................72
Checking for Hundreds ...........................................................................................................................73
Using the {n,m} Syntax ............................................................................................................................75
Checking for Tens and Ones ...................................................................................................................76
Verbose Regular Expressions ..................................................................................................................78
Case Study: Parsing Phone Numbers .....................................................................................................80
Further Reading Online...........................................................................................................................85
■Chapter 6: Closures and Generators................................................................................... 87
I Know, Let’s Use Regular Expressions!..................................................................................................88
A List of Functions ...................................................................................................................................90
A List of Patterns ......................................................................................................................................92
A File of Patterns ......................................................................................................................................95
Generators................................................................................................................................................96
A Fibonacci Generator.............................................................................................................................98
A Plural Rule Generator...........................................................................................................................99
Further Reading Online.........................................................................................................................100
■Chapter 7: Classes and Iterators ...................................................................................... 101
Defining Classes.....................................................................................................................................102
The __init__() Method............................................................................................................................102
Instantiating Classes .............................................................................................................................103
Instance Variables .................................................................................................................................104
A Fibonacci Iterator ...............................................................................................................................105
A Plural Rule Iterator.............................................................................................................................107
Further Reading Online.........................................................................................................................112

vii
 ■ CONTENTS

■Chapter 8: Advanced Iterators ......................................................................................... 113

Finding All Occurrences of a Pattern....................................................................................................115
Finding the Unique Items in a Sequence .............................................................................................115
Making Assertions .................................................................................................................................117
Generator Expressions ..........................................................................................................................117
Calculating Permutations … the Lazy Way..........................................................................................119
Other Fun Stuff in the itertools Module ...............................................................................................120
A New Kind of String Manipulation......................................................................................................124
Evaluating Arbitrary Strings as Python Expressions ...........................................................................126
Putting It All Together ...........................................................................................................................130
Further Reading Online.........................................................................................................................130
■Chapter 9: Unit Testing..................................................................................................... 131
A Single Question...................................................................................................................................132
Halt and Catch Fire................................................................................................................................138
More Halting, More Fire ........................................................................................................................141
And One More Thing … .........................................................................................................................144
A Pleasing Symmetry.............................................................................................................................146
More Bad Input ......................................................................................................................................150
■Chapter 10: Refactoring ................................................................................................... 155
Handling Changing Requirements.......................................................................................................158
Refactoring.............................................................................................................................................162
Further Reading Online.........................................................................................................................166
■Chapter 11: Files .............................................................................................................. 167
Reading from Text Files.........................................................................................................................167
Character Encoding Rears Its Ugly Head ........................................................................................168
Stream Objects ..................................................................................................................................169
Reading Data from a Text File..........................................................................................................170
Closing Files ......................................................................................................................................172
Closing Files Automatically..............................................................................................................173
Reading Data One Line at a Time.....................................................................................................173
Writing to Text Files...............................................................................................................................175
Character Encoding Again ...............................................................................................................176
Binary Files.............................................................................................................................................177
Streams Objects from Nonfile Sources.................................................................................................178
Handling Compressed Files .............................................................................................................179
Standard Input, Output, and Error.......................................................................................................180
Redirecting Standard Output...........................................................................................................181
Further Reading Online.........................................................................................................................184
■Chapter 12: XML............................................................................................................... 185
A 5-Minute Crash Course in XML .........................................................................................................186

viii
 ■ CONTENTS

The Structure of an Atom Feed .............................................................................................................189

Parsing XML ...........................................................................................................................................191
Elements Are Lists..................................................................................................................................192
Attributes Are Dictionaries....................................................................................................................193
Searching for Nodes Within an XML Document..................................................................................194
Going Further with lxml ........................................................................................................................197
Generating XML .....................................................................................................................................199
Parsing Broken XML ..............................................................................................................................202
Further Reading Online.........................................................................................................................204
■Chapter 13: Serializing Python Objects ............................................................................ 205
A Quick Note About the Examples in this Chapter .........................................................................205
Saving Data to a Pickle File ...................................................................................................................206
Loading Data from a Pickle File............................................................................................................208
Pickling Without a File ..........................................................................................................................209
Bytes and Strings Rear Their Ugly Heads Again ..................................................................................210
Debugging Pickle Files ..........................................................................................................................210
Serializing Python Objects to be Read by Other Languages ...............................................................213
Saving Data to a JSON File ....................................................................................................................213
Mapping Python Datatypes to JSON ....................................................................................................215
Serializing Datatypes Unsupported by JSON.......................................................................................215
Loading Data from a JSON File .............................................................................................................220
Further Reading Online.........................................................................................................................222
■Chapter 14: HTTP Web Services ....................................................................................... 225
Features of HTTP ...................................................................................................................................226
Caching..............................................................................................................................................226
Last-Modified Checking ...................................................................................................................227
ETags..................................................................................................................................................228
Compression .....................................................................................................................................229
Redirects............................................................................................................................................229
How Not to Fetch Data Over HTTP.......................................................................................................230
What’s On the Wire?...............................................................................................................................231
Introducing httplib2 ..............................................................................................................................234
Caching with httplib2 .......................................................................................................................237
Handling Last-Modified and ETag Headers with httplib2 .............................................................240
Handling Compression with httplib2..............................................................................................242
Handling Redirects with httplib2 ....................................................................................................243
Beyond HTTP GET .................................................................................................................................246
Beyond HTTP POST...............................................................................................................................250
Further Reading Online.........................................................................................................................252
■Chapter 15: Case Study: Porting chardet to Python 3 ...................................................... 253
What Is Character Encoding Auto-Detection? .....................................................................................253

ix
 ■ CONTENTS

Why Auto-Detection Is Difficult.......................................................................................................253

Auto-Encoding Algorithms...............................................................................................................254
Introducing the chardet Module ..........................................................................................................254
UTF-n with a BOM ............................................................................................................................254
Escaped Encodings ...........................................................................................................................254
Multibyte Encodings.........................................................................................................................255
Single-Byte Encodings......................................................................................................................255
windows-1252 ...................................................................................................................................256
Running 2to3..........................................................................................................................................256
A Short Digression Into Multi-File Modules ........................................................................................259
Fixing What 2to3 Can’t ..........................................................................................................................261
False Is Invalid Syntax ......................................................................................................................261
No Module Named Constants ..........................................................................................................262
Name 'file' Is Not Defined.................................................................................................................263
Can’t Use a String Pattern on a Bytes-Like Object..........................................................................264
Can’t Convert 'bytes' Object to str Implicitly ..................................................................................266
Unsupported Operand type(s) for +: 'int' and 'bytes' .....................................................................269
ord() Expected String of Length 1, but int Found ...........................................................................270
Unorderable Types: int() >= str() .....................................................................................................273
Global Name 'reduce' Is not Defined...............................................................................................275
Lessons Learned.....................................................................................................................................277
■Chapter 16: Packaging Python Libraries.......................................................................... 279
Things Distutils Can’t Do for You .........................................................................................................280
Directory Structure................................................................................................................................281
Writing Your Setup Script.................................................................................................................282
Classifying Your Package ......................................................................................................................284
Examples of Good Package Classifiers ............................................................................................284
Checking Your Setup Script for Errors .................................................................................................287
Creating a Source Distribution .............................................................................................................287
Creating a Graphical Installer...............................................................................................................289
Building Installable Packages for Other Operating Systems .........................................................290
Adding Your Software to the Python Package Index ...........................................................................291
The Many Possible Futures of Python Packaging................................................................................292
Further Reading Online.........................................................................................................................293
■Appendix A: Porting Code to Python 3 with 2to3 ............................................................. 295
print Statement......................................................................................................................................295
Unicode String Literals..........................................................................................................................296
unicode() Global Function ....................................................................................................................296
long Datatype.........................................................................................................................................297
<> Comparison ......................................................................................................................................297
has_key() Dictionary Method................................................................................................................298

x
 ■ CONTENTS

Dictionary Methods that Return Lists ..................................................................................................299

Renamed or Reorganized Modules.......................................................................................................299
http.....................................................................................................................................................300
urllib ..................................................................................................................................................300
dbm....................................................................................................................................................301
xmlrpc................................................................................................................................................302
Other Modules ..................................................................................................................................302
Relative Imports Within a Package.......................................................................................................304
next() Iterator Method...........................................................................................................................305
filter() Global Function..........................................................................................................................306
map() Global Function ..........................................................................................................................306
reduce() Global Function ......................................................................................................................307
apply() Global Function ........................................................................................................................308
intern() Global Function .......................................................................................................................308
exec Statement.......................................................................................................................................309
execfile Statement .................................................................................................................................309
repr Literals (Backticks) ........................................................................................................................310
try...except Statement............................................................................................................................310
raise Statement ......................................................................................................................................312
throw Method on Generators................................................................................................................312
xrange() Global Function ......................................................................................................................313
raw_input() and input() Global Functions...........................................................................................314
func_* Function Attributes....................................................................................................................314
xreadlines() I/O Method ........................................................................................................................315
lambda Functions that Take a Tuple Instead of Multiple Parameters...............................................316
Special Method Attributes.....................................................................................................................317
__nonzero__ Special Method ................................................................................................................317
Octal Literals ..........................................................................................................................................318
[Link] ..............................................................................................................................................318
callable() Global Function.....................................................................................................................319
zip() Global Function.............................................................................................................................319
StandardError Exception ......................................................................................................................319
types Module Constants ........................................................................................................................320
isinstance() Global Function.................................................................................................................321
basestring Datatype...............................................................................................................................321
itertools Module.....................................................................................................................................322
sys.exc_type, sys.exc_value, sys.exc_traceback ...................................................................................322
List Comprehensions Over Tuples........................................................................................................323
[Link]() Function...........................................................................................................................323
Metaclasses ............................................................................................................................................323
Matters of Style ......................................................................................................................................324
set() Literals (Explicit).......................................................................................................................324

xi
 ■ CONTENTS

buffer() Global Function (Explicit) ..................................................................................................324

Whitespace Around Commas (Explicit) ..........................................................................................325
Common Idioms (Explicit)...............................................................................................................325
■Appendix B: Special Method Names................................................................................. 327
Basics ......................................................................................................................................................327
Classes that Act Like Iterators ...............................................................................................................328
Computed Attributes .............................................................................................................................328
Classes that Act Like Functions.............................................................................................................331
Classes that Act Like Sequences............................................................................................................332
Classes that Act Like Dictionaries.........................................................................................................334
Classes that Act Like Numbers..............................................................................................................335
Classes that Can Be Compared .............................................................................................................339
Classes that Can Be Serialized ..............................................................................................................340
Classes that Can Be Used in a “with” Block .........................................................................................340
Really Esoteric Stuff...............................................................................................................................342
Further Reading Online.........................................................................................................................343
■Appendix C: Where to Go From Here................................................................................. 345
■Index ................................................................................................................................ 347

xii
Foreword

Seven years ago, I’d have looked at you incredulously and probably laughed if you had told me I would
be sitting here today writing the foreword to a book, much less the foreword to a programming book. Yet
here I am.
Seven years ago, I was just a test engineer with some scripting skills and a systems administration
background. I had little programming experience and even less passion for it.
One day, a soon-to-be-coworker of mine mentioned this “new” scripting language called Python.
He said it was easy to learn and might add to my skill set. I was wary because programmers seemed to be
so separated from my “real world” of tests and systems and users. But his description also made me
curious, so I visited the nearest bookstore and bought the first book on the subject I found.
The book I bought was the original Dive Into Python, by Mark Pilgrim. I have to believe that I am not
the only person who can say, without exaggeration, that Mark’s book changed my life and career forever.
The combination of Mark’s book, his passion for Python, his presentation of the material, and even
the Python (the language itself) fundamentally altered the way I thought. The combination drove me not
just to read “yet another book about tech stuff”; it drove me to code, to represent my ideas in a
completely new and exciting way. Mark’s passion for the language inspired me with a newfound
passion.
Now, seven years later, I’m a contributor to the Python standard library–an active community
member–and I teach the language to as many people as I can. I use it in my free time, I use it at my job,
and I contribute to it in between my daughter’s naps. Dive Into Python–and Python itself–changed me.
Python is neither the prettiest nor most flexible language out there. But it is clean, simple, and
powerful. Its elegance lies in its simplicity and its practicality. Its flexibility enables you (or anyone) to
get something–anything–done simply by “keeping out of your way.”
I've said for some time the beauty of Python is that it scales “up.” It is useful for someone who wants
only to do some math or write a simple script. And it is equally useful for programmers who want to
create large-scale systems, web frameworks, and multimillion dollar video-sharing sites.
Python has not been without its warts, though. Building a language is, at least in my mind, much
like learning to program. It’s an evolutionary process where you constantly have to question the
decisions you’ve made and be willing to correct those decisions.
Python 3 admits to some of those mistakes with its new fixes, removing some of the old warts, while
also possibly introducing some new ones. Python 3 shows a self-awareness and willingness to evolve in
much-needed ways you don't see in a lot of things.
Python 3 does not redefine, fundamentally alter, or suddenly invalidate all the Python you knew
before. Rather, it takes something that is time-proven and battle-worn and improves on it in rational,
practical ways.

xiii
 ■ FOREWORD

Python 3 doesn’t represent the end of the evolution of the language. New features, syntax, and
libraries continue to be added; and it will probably be added, tweaked, and removed for as long as
Python carries on.
Python 3 is simply a cleaner, more evolved platform for you, the reader, to get things done with.
Like Python 3, Dive Into Python 3 represents the evolution of something that was already very good
becoming something even better. Mark’s passion, wit, and engaging style are still there; and the material
has been expanded, improved, and updated. But like Python 3 itself, version 3 of this series
fundamentally remains the thing that originally gave me such a passion for programming.
Python’s simplicity is infectious. The passion of its community, not to mention the passion with
which the language is created and maintained, remains astounding.
I hope Mark’s passion, and Python itself, inspires you as it did me seven years ago. I hope you find
Python, and Python 3, to be as practical and powerful as the hundreds of thousands of programmers and
companies that use it across the world.
Jesse Noller
Python Developer

xiv
About the Author

■By day, Mark Pilgrim is a developer advocate for open source and open
standards. By night, he is a husband and father who lives in North Carolina with
his wife, his two sons, and his big, slobbery dog. He spends his copious free time
sunbathing, skydiving, and making up autobiographical information.

xv
 ■ FOREWORD

About the Technical Reviewer

■Simon Willison is a speaker, writer, developer and all-around web technology

enthusiast. Simon works for Guardian News and Media as a technical architect for
both [Link] and the recently launched Guardian Developer Network.
Before joining the Guardian Simon worked as a consultant for clients that
included the BBC, Automattic, and GCap Media. He is a past member of Yahoo!'s
Technology Development team (his projects included the initial prototype of
FireEagle, Yahoo!'s location broker API). Prior to Yahoo!, he worked at the
Lawrence Journal-World, an award winning local newspaper in Kansas.
Simon is a co-creator of the Django web framework, and a passionate
advocate for Open Source and standards-based development. He maintains a
popular web development weblog at [Link]

xvi
Acknowledgments

The author would like to thank his wife for her never-ending support and encouragement, without
which this book would still be an item on an ever-growing wish list.
Thank you to Raymond Hettinger for relicensing his alphametics solver so I could use it as the basis
for Chapter 8.
Thank you to Jesse Noller for patiently explaining so many things to me at PyCon 2009, so that I
could explain them to everyone else.
Finally, thank you to the many people who gave me feedback during the public writing process,
especially Giulio Piancastelli, Florian Wollenschein, and all the good people of [Link].

xvii
 ■ FOREWORD

Installing Python

Welcome to Python 3. Let’s dive in. In this chapter, you’ll install the version of Python 3 that’s right
for you.

Which Python Is Right for You?

The first thing you need to do with Python is install it. Or do you?
If you’re using an account on a hosted server, your Internet service provider (ISP) might have
already installed Python 3. If you’re running Linux at home, you might already have Python 3, too. Most
popular GNU/Linux distributions come with Python 2 in the default installation; a small but growing
number of distributions also include Python 3. (As you’ll see in this chapter, you can have more than one
version of Python installed on your computer.) Mac OS X includes a command-line version of Python 2,
but (as of this writing) it does not include Python 3. Microsoft Windows does not come with any version
of Python. But don’t despair! You can point and click your way through installing Python, regardless of
which operating system you have.
The easiest way to check for Python 3 on your Linux or Mac OS X system is to get to a command line.
On Linux, look in your Applications menu for a program called Terminal. (It might be in a submenu such
as Accessories or System.) On Mac OS X, there is an application called [Link] in your
/Application/Utilities/ folder.
Once you’re at a command-line prompt, just type python3 (all lowercase, no spaces) and see what
happens. On my home Linux system, Python 3 is already installed, and this command gets me into the
Python interactive shell, as shown in Listing 0-1.

Listing 0-1. Python Interactive Shell

mark@atlantis:~$ python3
Python 3.0.1+ (r301:69556, Apr 15 2009, [Link])
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

Type exit() and press Enter to exit the Python interactive shell.
My web hosting provider also runs Linux and provides command-line access, but as Listing 0-2
shows, my server does not have Python 3 installed. (Boo!)

xviii
 ■ INSTALLING PYTHON

Listing 0-2. Python Interactive Shell

mark@manganese:~$ python3
bash: python3: command not found

So back to the question that started this section: “Which Python is right for you?” The answer is
simple: whichever one runs on the computer you already have.

Installing on Microsoft Windows

Windows comes in two architectures these days: 32-bit and 64-bit. Of course, there are lots of different
versions of WindowsXP, Vista, and Windows 7but Python runs on all of them. The more important
distinction is 32-bit versus 64-bit. If you have no idea what architecture you’re running, it’s probably
32-bit.
Visit [Link] and download the appropriate Python 3 Windows installer for
your architecture. Your choices will look something like these:
• Python 3.1 Windows installer (Windows binarydoes not include source)
• Python 3.1 Windows AMD64 installer (Windows AMD64 binarydoes not include
source)
I don’t want to include direct download links here because minor updates of Python happen all the
time and I don’t want to be responsible for you missing important updates. You should always install the
most recent version of Python 3.x unless you have some esoteric reason not to.
Once your download is complete, double-click the .msi file. Windows will display a security alert
because you’re about to be running executable code. The official Python installer is digitally signed by
the Python Software Foundation, the nonprofit corporation that oversees Python development. Don’t
accept imitations!

xix
 ■ INSTALLING PYTHON

Click the Run button to launch the Python 3 installer.

The first question the installer will ask you is whether you want to install Python 3 for all users or just for
you. The default choice is “Install for all users,” which is the best choice unless you have a good reason to
choose otherwise. (One reason why you might want to choose “Install just for me” is that you are
installing Python on your company’s computer and you don’t have administrative rights on your
Windows account. But then why are you installing Python without permission from your company’s
Windows administrator? Don’t get me in trouble here!)

xx
 ■ INSTALLING PYTHON

Click the Next button to accept your choice of installation type.

Next, the installer will prompt you to choose a destination directory. The default for all versions of
Python 3.1.x is C:\Python31\, which should work well for most users unless you have a specific reason to
change it. If you maintain a separate drive letter for installing applications, you can browse to it using
the embedded controls or simply type the pathname in the box below. You are not limited to installing
Python on the C: drive; you can install it on any drive, in any folder.

xxi
 ■ INSTALLING PYTHON

Click the Next button to accept your choice of destination directory.

Although the next page looks complicated, it’s not really difficult. Like many installers, you have the
option not to install every single component of Python 3. If disk space is especially tight, you can exclude
certain components.
• Register Extensions allows you to double-click Python scripts (.py files) and run
them (recommended but not required). This option doesn’t require any disk
space, so there is little point in excluding it.
• Tcl/Tk is the graphics library used by the Python shell, which you will use
throughout this book. I strongly recommend keeping this option.
• Documentation installs a help file that contains much of the information on
[Link] This option is recommended if you are on dialup or have
limited Internet access.
• Utility Scripts includes the [Link] script, which you’ll learn about later in this
book. It is required if you want to learn about migrating existing Python 2 code to
Python 3. If you have no existing Python 2 code, you can skip this option.
• Test suite is a collection of scripts used to test the Python interpreter. You will not
use it in this book, nor have I ever used it in the course of programming in Python.
Completely optional.

xxii
 ■ INSTALLING PYTHON

If you don’t know how much disk space you have, click the Disk Usage button. The installer will list
your drive letters, compute how much space is available on each drive, and calculate how much would
be left after installation.

xxiii
 ■ INSTALLING PYTHON

Click the OK button to return to the customization page. If you decide to exclude an option, select
the drop-down button before the option and select “Entire feature will be unavailable”. For example,
excluding the Test suite will save you a whopping 7908KB of disk space.

xxiv
 ■ INSTALLING PYTHON

Click the Next button to accept your choice of options.

xxv
 ■ INSTALLING PYTHON

The installer will copy all the necessary files to your chosen destination directory. (This happens so
quickly, I had to try it three times to even get a screenshot of it!)

xxvi
 ■ INSTALLING PYTHON

Click the Finish button to exit the installer.

xxvii
 ■ INSTALLING PYTHON

In your Start menu, there should be a new item called Python 3.1, in which you find a program
called IDLE. Select this item to run the interactive Python shell.

Installing on Mac OS X
All modern Macintosh computers use the Intel chip (as most Windows PCs do). Older Macs used
PowerPC chips. You don’t need to understand the difference because there’s just one Mac Python
installer for all Macs.
Visit [Link] and download the Mac installer. It will be called something like
Python 3.1 Mac Installer Disk Image, although the version number might vary. Be sure to download
version 3.x, not 2.x.
Your browser should automatically mount the disk image and open a Finder window to show you
the contents. (If this doesn’t happen, you’ll need to find the disk image in your Downloads folder and
double-click to mount it. It will be named something like [Link].) The disk image contains a
number of text files ([Link], [Link], [Link]) and the actual installer package, [Link].

xxviii
 ■ INSTALLING PYTHON

Double-click the [Link] installer package to launch the Mac Python installer.

The first page of the installer gives a brief description of Python itself. It then refers you to the
[Link] file (which you didn’t read, did you?) for more details.

xxix
 ■ INSTALLING PYTHON

Click the Continue button to move along.

The next page actually contains some important information: Python requires Mac OS X 10.3 or
later. If you are still running Mac OS X 10.2, you should upgrade. Apple no longer provides security
updates for your operating system, and your computer is probably at risk if you ever go online. Also, you
can’t run Python 3.

xxx
 ■ INSTALLING PYTHON

Click the Continue button to advance.

Like all good installers, the Python installer displays the software license agreement. Python is open
source and its license is approved by the Open Source Initiative. Python has had a number of owners
and sponsors throughout its history, each of which has left its mark on the software license. But the end
result is this: Python is open source, and you are allowed use it on any platform, for any purpose,
without fee or obligation of reciprocity.

xxxi
 ■ INSTALLING PYTHON

Click the Continue button once again.

Because of quirks in the standard Apple installer framework, you must “agree” to the software
license to complete the installation. Because Python is open source, you are really “agreeing” that the
license is granting you additional rights instead of taking them away.

xxxii
 ■ INSTALLING PYTHON

Click the Agree button to continue.

The next screen allows you to change your install location. You must install Python on your boot
drive, but because of limitations of the installer, it does not enforce it. In truth, I have never had the need
to change the install location.

xxxiii
 ■ INSTALLING PYTHON

From this screen, you can also customize the installation to exclude certain features. If you want to
do this, click the Customize button; otherwise click the Install button.

If you choose Custom Install, the installer will present you with the following list of features:
• Python Framework is the guts of Python, and it is both selected and disabled
because it must be installed.
• GUI Applications includes IDLE, the graphical Python shell that you will use
throughout this book. I strongly recommend keeping this option selected.
• “UNIX command-line tools” includes the command-line python3 application. I
strongly recommend keeping this option, too.
• Python Documentation contains much of the information on
[Link] This option is recommended if you are on dialup or have
limited Internet access.
• “Shell profile updater” controls whether to update your shell profile (used in
[Link]) to ensure that this version of Python is on the search path of your
shell. You probably don’t need to change it.

xxxiv
 ■ INSTALLING PYTHON

• “Fix system Python” should not be changed. It tells your Mac to use Python 3 as
the default Python for all scripts, including built-in system scripts from Apple.
This would be very bad because most of those scripts are written for Python 2, and
they would fail to run properly under Python 3.
Click the Install button to continue.

Because it installs system-wide frameworks and binaries in /usr/local/bin/, the installer will ask
you for an administrative password. There is no way to install Mac Python without administrator
privileges.

xxxv
 ■ INSTALLING PYTHON

After supplying the password, click the OK button to begin the installation.

xxxvi
 ■ INSTALLING PYTHON

The installer will display a progress meter while it installs the features you’ve selected.

Assuming that all went well, the installer will show you a big green check mark to tell you that the
installation completed successfully.

xxxvii
 ■ INSTALLING PYTHON

Click the Close button to exit the installer.

Assuming that you didn’t change the install location, you can find the newly installed files in the
Python 3.1 folder within your /Applications folder. The most important piece is IDLE, the graphical
Python shell.

xxxviii
 ■ INSTALLING PYTHON

Double-click IDLE to launch the graphical Python shell.

xxxix
 ■ INSTALLING PYTHON

The Python shell is where you will spend most of your time exploring Python. Examples throughout
this book will assume that you can find your way into the Python shell.

Installing on Ubuntu Linux

Modern Linux distributions are backed by vast repositories of precompiled applications, ready to install.
The exact details vary by distribution. In Ubuntu Linux, the easiest way to install Python 3 is through the
Add/Remove application in your Applications menu.
When you first launch the Add/Remove application, it will show you a list of preselected applications
in different categories. Some are already installed; most are not. Because the repository contains more
than 10,000 applications, there are different filters you can apply to see small parts of the repository. The
default filter (“Canonical-maintained applications”) is a small subset of the total number of applications
that are officially supported by Canonical, the company that creates and maintains Ubuntu Linux.

xl
 ■ INSTALLING PYTHON

Python 3 is not maintained by Canonical, so the first step is to drop down this filter menu and select
“All Open Source applications”.

Once you’ve widened the filter to include all open source applications, use the Search box
immediately after the filter menu to search for Python 3.

xli
 ■ INSTALLING PYTHON

xlii
 ■ INSTALLING PYTHON

Now the list of applications narrows to just those matching Python 3. You’ll check two packages.
The first is Python (v3.0), which contains the Python interpreter.

The second package you want is immediately above: IDLE (using Python-3.0). This is a graphical
Python shell that you will use throughout this book.

xliii
 ■ INSTALLING PYTHON

After you’ve checked those two packages, click the Apply Changes button to continue.

The package manager will ask you to confirm that you want to add both IDLE (using Python-3.0) and
Python (v3.0).

xliv
 ■ INSTALLING PYTHON

Click the Apply button to continue.

The package manager will show you a progress meter while it downloads the necessary packages
from Canonical’s Internet repository.

xlv
 ■ INSTALLING PYTHON

Once the packages are downloaded, the package manager will automatically begin installing them.

If all went well, the package manager will confirm that both packages were successfully installed.
From here, you can double-click IDLE to launch the Python shell or click the Close button to exit the
package manager.
You can always relaunch the Python shell by going to your Applications menu, choosing the
Programming submenu, and selecting IDLE.

The Python shell is where you will spend most of your time exploring Python. Examples throughout
this book will assume that you can find your way into the Python shell.

xlvi
 ■ INSTALLING PYTHON

Installing on Other Platforms

Python 3 is available on a number of different platforms. In particular, it is available in virtually every
Linux-, BSD-, and Solaris-based distribution. For example, RedHat Linux uses the yum package manager;
FreeBSD has its ports and packages collection; Solaris has pkgadd and friends. A quick web search for
“Python 3” + “your operating system” will tell you whether a Python 3 package is available and how to
install it.

Using the Python Shell

The Python shell is where you can explore Python syntax, get interactive help for commands, and debug
short programs. The graphical Python shell (named IDLE) also contains a decent text editor that
supports Python syntax coloring and integrates with the Python shell. If you don’t already have a favorite
text editor, you should give IDLE a try.
First things first. The Python shell itself is an amazing interactive playground. Throughout this book,
you’ll see examples like the one shown in Listing 0-3.

Listing 0-3. Evaluating Expressions in the Python Interactive Shell

>>> 1 + 1
2

The three angle brackets, >>>, denote the Python shell prompt. Don’t type that part. That’s just to let
you know that this example is meant to be followed in the Python shell.
1 + 1 is the part you type. You can type any valid Python expression or command in the Python
shell. Don’t be shy; it won’t bite! The worst that will happen is you’ll get an error message. Commands

xlvii
 ■ INSTALLING PYTHON

get executed immediately (after you press Enter), expressions get evaluated immediately, and the
Python shell prints out the result.
2 is the result of evaluating this expression. As it happens, 1 + 1 is a valid Python expression. The
result, of course, is 2.
Let’s try another one (see Listing 0-4).

Listing 0-4. Hello World!

>>> print('Hello world!')

Hello world!

Pretty simple, no? But there are more things you can do in the Python shell. If you ever get
stuckyou can’t remember a command or you can’t remember the proper arguments to pass a certain
functionyou can get interactive help in the Python shell. Just type help() and press Enter, as shown in
Listing 0-5.

Listing 0-5. Help?

>>> help
Type help() for interactive help, or help(object) for help about object.

There are two modes of help. You can get help about a single object, which just prints out the
documentation and returns you to the Python shell prompt. You can also enter help mode, where
instead of evaluating Python expressions, you just type keywords or command names and it will print
out whatever it knows about that command.
To enter the interactive help mode, type help() and press Enter, as shown in Listing 0-6.

Listing 0-6. Interactive Help Mode

>>> help()
Welcome to Python 3.1! This is the online help utility.

If this is your first time using Python, you should definitely check out
the tutorial on the Internet at [Link]

Enter the name of any module, keyword, or topic to get help on writing
Python programs and using Python modules. To quit this help utility and
return to the interpreter, just type "quit".

To get a list of available modules, keywords, or topics, type "modules",

"keywords", or "topics". Each module also comes with a one-line summary
of what it does; to list the modules whose summaries contain a given word
such as "spam", type "modules spam".

help>

Note that the prompt changes from >>> to help>. This reminds you that you’re in the interactive
help mode. Now you can enter any keyword, command, module name, function namepretty much
anything Python understandsand read documentation on it (see Listing 0-7).

xlviii
 ■ INSTALLING PYTHON

Listing 0-7. Interactive Help Mode

help> print (1)

Help on built-in function print in module builtins:

print(...)
print(value, ..., sep=' ', end='\n', file=[Link])

Prints the values to a stream, or to [Link] by default.

Optional keyword arguments:
file: a file-like object (stream); defaults to the current [Link].
sep: string inserted between values, default a space.
end: string appended after the last value, default a newline.

help> PapayaWhip (2)

no Python documentation found for 'PapayaWhip'

help> quit (3)

You are now leaving help and returning to the Python interpreter.
If you want to ask for help on a particular object directly from the
interpreter, you can type "help(object)". Executing "help('string')"
has the same effect as typing a particular string at the help> prompt.
>>> (4)

The following notes refer to the numbered lines in Listing 0-7:

1. To get documentation on the print() function, just type print and press Enter.
The interactive help mode will display something akin to a man page: the
function name, a brief synopsis, the function’s arguments and their default
values, and so on. If the documentation seems opaque to you, don’t panic.
You’ll learn more about all these concepts in the next few chapters.
2. Of course, the interactive help mode doesn’t know everything. If you type
something that isn’t a Python command, module, function, or other built-in
keyword, the interactive help mode will just shrug its virtual shoulders.
3. To quit the interactive help mode, type quit and then press Enter.
4. The prompt changes back to >>> to signal that you’ve left the interactive help
mode and returned to the Python shell.
IDLE, the graphical Python shell, also includes a Python—aware text editor. You’ll see how to use it in
the next chapter.

Python Editors and IDEs

IDLE is not the only game in town when it comes to writing programs in Python. While IDLE is useful to
get started with learning the language itself, many developers prefer other text editors or integrated
development environments (IDEs). I won’t cover them here, but the Python community maintains a list
of Python-aware editors at [Link] that covers a wide range of
supported platforms and software licenses.

xlix
 ■ INSTALLING PYTHON

You might also want to check out the list of Python-aware IDEs at
[Link] although few of them support
Python 3 yet. One that does is PyDev ([Link] a plugin for Eclipse
([Link] that turns Eclipse into a full-fledged Python IDE. Both Eclipse and PyDev are
cross-platform and open source.
On the commercial front, there is ActiveState’s Komodo IDE at
[Link] It has per-user licensing, but students can get a discount, and a
free time-limited trial version is available.
I’ve been programming in Python for nine years, and I edit my Python programs in GNU Emacs and
debug them in the command-line Python shell. There’s no right or wrong way to develop in Python. Find
a way that works for you!