Table of Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxxi

Part I. Getting Started

1. A Python Q&A Session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Why Do People Use Python? 3
Software Quality 4
Developer Productivity 5
Is Python a “Scripting Language”? 5
OK, but What’s the Downside? 7
Who Uses Python Today? 7
What Can I Do with Python? 9
Systems Programming 9
GUIs 9
Internet Scripting 10
Component Integration 10
Database Programming 11
Rapid Prototyping 11
Numeric and Scientific Programming 11
Gaming, Images, Serial Ports, XML, Robots, and More 12
How Is Python Supported? 12
What Are Python’s Technical Strengths? 13
It’s Object-Oriented 13
It’s Free 13
It’s Portable 14
It’s Powerful 15
It’s Mixable 16
It’s Easy to Use 16
It’s Easy to Learn 17
It’s Named After Monty Python 17
How Does Python Stack Up to Language X? 17

vii
 Chapter Summary 18
Test Your Knowledge: Quiz 19
Test Your Knowledge: Answers 19

2. How Python Runs Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Introducing the Python Interpreter 23
Program Execution 24
The Programmer’s View 24
Python’s View 26
Execution Model Variations 29
Python Implementation Alternatives 29
Execution Optimization Tools 30
Frozen Binaries 32
Other Execution Options 33
Future Possibilities? 33
Chapter Summary 34
Test Your Knowledge: Quiz 34
Test Your Knowledge: Answers 34

3. How You Run Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

The Interactive Prompt 35
Running Code Interactively 37
Why the Interactive Prompt? 38
Using the Interactive Prompt 39
System Command Lines and Files 41
A First Script 42
Running Files with Command Lines 43
Using Command Lines and Files 44
Unix Executable Scripts (#!) 46
Clicking File Icons 47
Clicking Icons on Windows 47
The input Trick 49
Other Icon-Click Limitations 50
Module Imports and Reloads 51
The Grander Module Story: Attributes 53
import and reload Usage Notes 56
Using exec to Run Module Files 57
The IDLE User Interface 58
IDLE Basics 58
Using IDLE 60
Advanced IDLE Tools 62
Other IDEs 63
Other Launch Options 64

viii | Table of Contents

 Embedding Calls 64
Frozen Binary Executables 65
Text Editor Launch Options 65
Still Other Launch Options 66
Future Possibilities? 66
Which Option Should I Use? 66
Chapter Summary 68
Test Your Knowledge: Quiz 68
Test Your Knowledge: Answers 69
Test Your Knowledge: Part I Exercises 70

Part II. Types and Operations

4. Introducing Python Object Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

Why Use Built-in Types? 76
Python’s Core Data Types 77
Numbers 78
Strings 80
Sequence Operations 80
Immutability 82
Type-Specific Methods 82
Getting Help 84
Other Ways to Code Strings 85
Pattern Matching 85
Lists 86
Sequence Operations 86
Type-Specific Operations 87
Bounds Checking 87
Nesting 88
Comprehensions 88
Dictionaries 90
Mapping Operations 90
Nesting Revisited 91
Sorting Keys: for Loops 93
Iteration and Optimization 94
Missing Keys: if Tests 95
Tuples 96
Why Tuples? 97
Files 97
Other File-Like Tools 99
Other Core Types 99
How to Break Your Code’s Flexibility 100

Table of Contents | ix
 User-Defined Classes 101
And Everything Else 102
Chapter Summary 103
Test Your Knowledge: Quiz 103
Test Your Knowledge: Answers 104

5. Numeric Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

Numeric Type Basics 105
Numeric Literals 106
Built-in Numeric Tools 108
Python Expression Operators 108
Numbers in Action 113
Variables and Basic Expressions 113
Numeric Display Formats 115
Comparisons: Normal and Chained 116
Division: Classic, Floor, and True 117
Integer Precision 121
Complex Numbers 122
Hexadecimal, Octal, and Binary Notation 122
Bitwise Operations 124
Other Built-in Numeric Tools 125
Other Numeric Types 127
Decimal Type 127
Fraction Type 129
Sets 133
Booleans 139
Numeric Extensions 140
Chapter Summary 141
Test Your Knowledge: Quiz 141
Test Your Knowledge: Answers 141

6. The Dynamic Typing Interlude . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

The Case of the Missing Declaration Statements 143
Variables, Objects, and References 144
Types Live with Objects, Not Variables 145
Objects Are Garbage-Collected 146
Shared References 148
Shared References and In-Place Changes 149
Shared References and Equality 151
Dynamic Typing Is Everywhere 152
Chapter Summary 153
Test Your Knowledge: Quiz 153
Test Your Knowledge: Answers 154

x | Table of Contents
7. Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
String Literals 157
Single- and Double-Quoted Strings Are the Same 158
Escape Sequences Represent Special Bytes 158
Raw Strings Suppress Escapes 161
Triple Quotes Code Multiline Block Strings 162
Strings in Action 163
Basic Operations 164
Indexing and Slicing 165
String Conversion Tools 169
Changing Strings 171
String Methods 172
String Method Examples: Changing Strings 174
String Method Examples: Parsing Text 176
Other Common String Methods in Action 177
The Original string Module (Gone in 3.0) 178
String Formatting Expressions 179
Advanced String Formatting Expressions 181
Dictionary-Based String Formatting Expressions 182
String Formatting Method Calls 183
The Basics 184
Adding Keys, Attributes, and Offsets 184
Adding Specific Formatting 185
Comparison to the % Formatting Expression 187
Why the New Format Method? 190
General Type Categories 193
Types Share Operation Sets by Categories 194
Mutable Types Can Be Changed In-Place 194
Chapter Summary 195
Test Your Knowledge: Quiz 195
Test Your Knowledge: Answers 196

8. Lists and Dictionaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

Lists 197
Lists in Action 200
Basic List Operations 200
List Iteration and Comprehensions 200
Indexing, Slicing, and Matrixes 201
Changing Lists In-Place 202
Dictionaries 207
Dictionaries in Action 209
Basic Dictionary Operations 209
Changing Dictionaries In-Place 210

Table of Contents | xi
 More Dictionary Methods 211
A Languages Table 212
Dictionary Usage Notes 213
Other Ways to Make Dictionaries 216
Dictionary Changes in Python 3.0 217
Chapter Summary 223
Test Your Knowledge: Quiz 224
Test Your Knowledge: Answers 224

9. Tuples, Files, and Everything Else . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225

Tuples 225
Tuples in Action 227
Why Lists and Tuples? 229
Files 229
Opening Files 230
Using Files 231
Files in Action 232
Other File Tools 238
Type Categories Revisited 239
Object Flexibility 241
References Versus Copies 241
Comparisons, Equality, and Truth 244
Python 3.0 Dictionary Comparisons 246
The Meaning of True and False in Python 246
Python’s Type Hierarchies 248
Type Objects 249
Other Types in Python 250
Built-in Type Gotchas 251
Assignment Creates References, Not Copies 251
Repetition Adds One Level Deep 252
Beware of Cyclic Data Structures 252
Immutable Types Can’t Be Changed In-Place 253
Chapter Summary 253
Test Your Knowledge: Quiz 254
Test Your Knowledge: Answers 254
Test Your Knowledge: Part II Exercises 255

Part III. Statements and Syntax

10. Introducing Python Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261

Python Program Structure Revisited 261
Python’s Statements 262

xii | Table of Contents

 A Tale of Two ifs 264
What Python Adds 264
What Python Removes 265
Why Indentation Syntax? 266
A Few Special Cases 269
A Quick Example: Interactive Loops 271
A Simple Interactive Loop 271
Doing Math on User Inputs 272
Handling Errors by Testing Inputs 273
Handling Errors with try Statements 274
Nesting Code Three Levels Deep 275
Chapter Summary 276
Test Your Knowledge: Quiz 276
Test Your Knowledge: Answers 277

11. Assignments, Expressions, and Prints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279

Assignment Statements 279
Assignment Statement Forms 280
Sequence Assignments 281
Extended Sequence Unpacking in Python 3.0 284
Multiple-Target Assignments 288
Augmented Assignments 289
Variable Name Rules 292
Expression Statements 295
Expression Statements and In-Place Changes 296
Print Operations 297
The Python 3.0 print Function 298
The Python 2.6 print Statement 300
Print Stream Redirection 302
Version-Neutral Printing 306
Chapter Summary 308
Test Your Knowledge: Quiz 308
Test Your Knowledge: Answers 308

12. if Tests and Syntax Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311

if Statements 311
General Format 311
Basic Examples 312
Multiway Branching 312
Python Syntax Rules 314
Block Delimiters: Indentation Rules 315
Statement Delimiters: Lines and Continuations 317
A Few Special Cases 318

Table of Contents | xiii

 Truth Tests 320
The if/else Ternary Expression 321
Chapter Summary 324
Test Your Knowledge: Quiz 324
Test Your Knowledge: Answers 324

13. while and for Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327

while Loops 327
General Format 328
Examples 328
break, continue, pass, and the Loop else 329
General Loop Format 329
pass 330
continue 331
break 331
Loop else 332
for Loops 334
General Format 334
Examples 335
Loop Coding Techniques 341
Counter Loops: while and range 342
Nonexhaustive Traversals: range and Slices 343
Changing Lists: range 344
Parallel Traversals: zip and map 345
Generating Both Offsets and Items: enumerate 348
Chapter Summary 349
Test Your Knowledge: Quiz 349
Test Your Knowledge: Answers 350

14. Iterations and Comprehensions, Part 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351

Iterators: A First Look 351
The Iteration Protocol: File Iterators 352
Manual Iteration: iter and next 354
Other Built-in Type Iterators 356
List Comprehensions: A First Look 358
List Comprehension Basics 359
Using List Comprehensions on Files 359
Extended List Comprehension Syntax 361
Other Iteration Contexts 362
New Iterables in Python 3.0 366
The range Iterator 367
The map, zip, and filter Iterators 368
Multiple Versus Single Iterators 369

xiv | Table of Contents

 Dictionary View Iterators 370
Other Iterator Topics 372
Chapter Summary 372
Test Your Knowledge: Quiz 372
Test Your Knowledge: Answers 373

15. The Documentation Interlude . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375

Python Documentation Sources 375
# Comments 376
The dir Function 376
Docstrings: __doc__ 377
PyDoc: The help Function 380
PyDoc: HTML Reports 383
The Standard Manual Set 386
Web Resources 387
Published Books 387
Common Coding Gotchas 387
Chapter Summary 389
Test Your Knowledge: Quiz 389
Test Your Knowledge: Answers 390
Test Your Knowledge: Part III Exercises 390

Part IV. Functions

16. Function Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395

Why Use Functions? 396
Coding Functions 396
def Statements 398
def Executes at Runtime 399
A First Example: Definitions and Calls 400
Definition 400
Calls 400
Polymorphism in Python 401
A Second Example: Intersecting Sequences 402
Definition 402
Calls 403
Polymorphism Revisited 403
Local Variables 404
Chapter Summary 404
Test Your Knowledge: Quiz 405
Test Your Knowledge: Answers 405

Table of Contents | xv
 17. Scopes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407
Python Scope Basics 407
Scope Rules 408
Name Resolution: The LEGB Rule 410
Scope Example 411
The Built-in Scope 412
The global Statement 414
Minimize Global Variables 415
Minimize Cross-File Changes 416
Other Ways to Access Globals 418
Scopes and Nested Functions 419
Nested Scope Details 419
Nested Scope Examples 419
The nonlocal Statement 425
nonlocal Basics 425
nonlocal in Action 426
Why nonlocal? 429
Chapter Summary 432
Test Your Knowledge: Quiz 433
Test Your Knowledge: Answers 434

18. Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435

Argument-Passing Basics 435
Arguments and Shared References 436
Avoiding Mutable Argument Changes 438
Simulating Output Parameters 439
Special Argument-Matching Modes 440
The Basics 441
Matching Syntax 442
The Gritty Details 443
Keyword and Default Examples 444
Arbitrary Arguments Examples 446
Python 3.0 Keyword-Only Arguments 450
The min Wakeup Call! 453
Full Credit 454
Bonus Points 455
The Punch Line... 456
Generalized Set Functions 456
Emulating the Python 3.0 print Function 457
Using Keyword-Only Arguments 459
Chapter Summary 460
Test Your Knowledge: Quiz 461
Test Your Knowledge: Answers 462

xvi | Table of Contents

19. Advanced Function Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463
Function Design Concepts 463
Recursive Functions 465
Summation with Recursion 465
Coding Alternatives 466
Loop Statements Versus Recursion 467
Handling Arbitrary Structures 468
Function Objects: Attributes and Annotations 469
Indirect Function Calls 469
Function Introspection 470
Function Attributes 471
Function Annotations in 3.0 472
Anonymous Functions: lambda 474
lambda Basics 474
Why Use lambda? 475
How (Not) to Obfuscate Your Python Code 477
Nested lambdas and Scopes 478
Mapping Functions over Sequences: map 479
Functional Programming Tools: filter and reduce 481
Chapter Summary 483
Test Your Knowledge: Quiz 483
Test Your Knowledge: Answers 483

20. Iterations and Comprehensions, Part 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485

List Comprehensions Revisited: Functional Tools 485
List Comprehensions Versus map 486
Adding Tests and Nested Loops: filter 487
List Comprehensions and Matrixes 489
Comprehending List Comprehensions 490
Iterators Revisited: Generators 492
Generator Functions: yield Versus return 492
Generator Expressions: Iterators Meet Comprehensions 497
Generator Functions Versus Generator Expressions 498
Generators Are Single-Iterator Objects 499
Emulating zip and map with Iteration Tools 500
Value Generation in Built-in Types and Classes 506
3.0 Comprehension Syntax Summary 507
Comprehending Set and Dictionary Comprehensions 507
Extended Comprehension Syntax for Sets and Dictionaries 508
Timing Iteration Alternatives 509
Timing Module 509
Timing Script 510
Timing Results 511

Table of Contents | xvii

 Timing Module Alternatives 513
Other Suggestions 517
Function Gotchas 518
Local Names Are Detected Statically 518
Defaults and Mutable Objects 520
Functions Without returns 522
Enclosing Scope Loop Variables 522
Chapter Summary 522
Test Your Knowledge: Quiz 523
Test Your Knowledge: Answers 523
Test Your Knowledge: Part IV Exercises 524

Part V. Modules

21. Modules: The Big Picture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529

Why Use Modules? 529
Python Program Architecture 530
How to Structure a Program 531
Imports and Attributes 531
Standard Library Modules 533
How Imports Work 533
1. Find It 534
2. Compile It (Maybe) 534
3. Run It 535
The Module Search Path 535
Configuring the Search Path 537
Search Path Variations 538
The [Link] List 538
Module File Selection 539
Advanced Module Selection Concepts 540
Chapter Summary 541
Test Your Knowledge: Quiz 541
Test Your Knowledge: Answers 542

22. Module Coding Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543

Module Creation 543
Module Usage 544
The import Statement 544
The from Statement 545
The from * Statement 545
Imports Happen Only Once 546
import and from Are Assignments 546

xviii | Table of Contents

 Cross-File Name Changes 547
import and from Equivalence 548
Potential Pitfalls of the from Statement 548
Module Namespaces 550
Files Generate Namespaces 550
Attribute Name Qualification 552
Imports Versus Scopes 552
Namespace Nesting 553
Reloading Modules 554
reload Basics 555
reload Example 556
Chapter Summary 558
Test Your Knowledge: Quiz 558
Test Your Knowledge: Answers 558

23. Module Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 561

Package Import Basics 561
Packages and Search Path Settings 562
Package __init__.py Files 563
Package Import Example 564
from Versus import with Packages 566
Why Use Package Imports? 566
A Tale of Three Systems 567
Package Relative Imports 569
Changes in Python 3.0 570
Relative Import Basics 570
Why Relative Imports? 572
The Scope of Relative Imports 574
Module Lookup Rules Summary 575
Relative Imports in Action 575
Chapter Summary 581
Test Your Knowledge: Quiz 582
Test Your Knowledge: Answers 582

24. Advanced Module Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 583

Data Hiding in Modules 583
Minimizing from * Damage: _X and __all__ 584
Enabling Future Language Features 584
Mixed Usage Modes: __name__ and __main__ 585
Unit Tests with __name__ 586
Using Command-Line Arguments with __name__ 587
Changing the Module Search Path 590
The as Extension for import and from 591

Table of Contents | xix