Python Cookbook | O'Reilly Media

Tell a friend

Python Cookbook, Second Edition By Alex Martelli, Anna Martelli Ravenscroft, David Ascher
March 2005
Pages: 844

| Table of Contents | Index | Sample Chapter | Colophon

Chapter 1 Text
1. Introduction
2. Processing a String One Character at a Time
3. Converting Between Characters and Numeric Codes
4. Testing Whether an Object Is String-like
5. Aligning Strings
6. Trimming Space from the Ends of a String
7. Combining Strings
8. Reversing a String by Words or Characters
9. Checking Whether a String Contains a Set of Characters
10. Simplifying Usage of Strings' translate Method
11. Filtering a String for a Set of Characters
12. Checking Whether a String Is Text or Binary
13. Controlling Case
14. Accessing Substrings
15. Changing the Indentation of a Multiline String
16. Expanding and Compressing Tabs
17. Interpolating Variables in a String
18. Interpolating Variables in a Stringin Python 2.4
19. Replacing Multiple Patterns in a Single Pass
20. Checking a String for Any of Multiple Endings
21. Handling International Text with Unicode
22. Converting Between Unicode and Plain Strings
23. Printing Unicode Charactersto Standard Output
24. Encoding Unicode Data for XML and HTML
25. Making Some Strings Case-Insensitive
26. Converting HTML Documents to Texton a Unix Terminal
Chapter 2 Files
1. Introduction
2. Reading from a File
3. Writing to a File
4. Searching and Replacing Text in a File
5. Reading a Specific Line from a File
6. Counting Lines in a File
7. Processing Every Word in a File
8. Using Random-Access Input/Output
9. Updating a Random-Access File
10. Reading Data from zip Files
11. Handling a zip File Inside a String
12. Archiving a Tree of Files into a Compressed tar File
13. Sending Binary Data to Standard Output Under Windows
14. Using a C++-like iostream Syntax
15. Rewinding an Input File to the Beginning
16. Adapting a File-like Object to a True File Object
17. Walking Directory Trees
18. Swapping One File Extension for Another Throughout a Directory Tree
19. Finding a File Given a Search Path
20. Finding Files Given a Search Path and a Pattern
21. Finding a File on the Python Search Path
22. Dynamically Changing the PythonSearch Path
23. Computing the Relative Path from One Directory to Another
24. Reading an Unbuffered Character in a Cross-Platform Way
25. Counting Pages of PDF Documents on Mac OS X
26. Changing File Attributes on Windows
27. Extracting Text from OpenOffice.org Documents
28. Extracting Text from Microsoft Word Documents
29. File Locking Using a Cross-Platform API
30. Versioning Filenames
31. Calculating CRC-64 Cyclic Redundancy Checks
Chapter 3 Time and Money
1. Introduction
2. Calculating Yesterday and Tomorrow
3. Finding Last Friday
4. Calculating Time Periods in a Date Range
5. Summing Durations of Songs
6. Calculating the Number of Weekdays Between Two Dates
7. Looking up Holidays Automatically
8. Fuzzy Parsing of Dates
9. Checking Whether Daylight Saving Time Is Currently in Effect
10. Converting Time Zones
11. Running a Command Repeatedly
12. Scheduling Commands
13. Doing Decimal Arithmetic
14. Formatting Decimals as Currency
15. Using Python as a Simple Adding Machine
16. Checking a Credit Card Checksum
17. Watching Foreign Exchange Rates
Chapter 4 Python Shortcuts
1. Introduction
2. Copying an Object
3. Constructing Lists with List Comprehensions
4. Returning an Element of a List If It Exists
5. Looping over Items and Their Indices in a Sequence
6. Creating Lists of Lists Without Sharing References
7. Flattening a Nested Sequence
8. Removing or Reordering Columnsin a List of Rows
9. Transposing Two-Dimensional Arrays
10. Getting a Value from a Dictionary
11. Adding an Entry to a Dictionary
12. Building a Dictionary Without Excessive Quoting
13. Building a Dict from a List of Alternating Keys and Values
14. Extracting a Subset of a Dictionary
15. Inverting a Dictionary
16. Associating Multiple Values with Each Key in a Dictionary
17. Using a Dictionary to Dispatch Methods or Functions
18. Finding Unions and Intersections of Dictionaries
19. Collecting a Bunch of Named Items
20. Assigning and Testing with One Statement
21. Using printf in Python
22. Randomly Picking Items with Given Probabilities
23. Handling Exceptions Within an Expression
24. Ensuring a Name Is Defined in a Given Module
Chapter 5 Searching and Sorting
1. Introduction
2. Sorting a Dictionary
3. Sorting a List of Strings Case-Insensitively
4. Sorting a List of Objects by an Attribute of the Objects
5. Sorting Keys or Indices Basedon the Corresponding Values
6. Sorting Strings with Embedded Numbers
7. Processing All of a List's Items in Random Order
8. Keeping a Sequence Ordered as Items Are Added
9. Getting the First Few Smallest Items of a Sequence
10. Looking for Items in a Sorted Sequence
11. Selecting the nth Smallest Element of a Sequence
12. Showing off quicksort in Three Lines
13. Performing Frequent Membership Tests on a Sequence
14. Finding Subsequences
15. Enriching the Dictionary Type with Ratings Functionality
16. Sorting Names and Separating Them by Initials
Chapter 6 Object-Oriented Programming
1. Introduction
2. Converting Among Temperature Scales
3. Defining Constants
4. Restricting Attribute Setting
5. Chaining Dictionary Lookups
6. Delegating Automatically as an Alternative to Inheritance
7. Delegating Special Methods in Proxies
8. Implementing Tuples with Named Items
9. Avoiding Boilerplate Accessors for Properties
10. Making a Fast Copy of an Object
11. Keeping References to Bound Methods Without Inhibiting Garbage Collection
12. Implementing a Ring Buffer
13. Checking an Instance for Any State Changes
14. Checking Whether an Object Has Necessary Attributes
15. Implementing the State Design Pattern
16. Implementing the "Singleton" Design Pattern
17. Avoiding the "Singleton" Design Pattern with the Borg Idiom
18. Implementing the Null Object Design Pattern
19. Automatically Initializing Instance Variables from _ _init_ _ Arguments
20. Calling a Superclass _ _init_ _ Method If It Exists
21. Using Cooperative Supercalls Concisely and Safely
Chapter 7 Persistence and Databases
1. Introduction
2. Serializing Data Using the marshal Module
3. Serializing Data Using the pickle and cPickle Modules
4. Using Compression with Pickling
5. Using the cPickle Module on Classes and Instances
6. Holding Bound Methods in a Picklable Way
7. Pickling Code Objects
8. Mutating Objects with shelve
9. Using the Berkeley DB Database
10. Accessing a MySQL Database
11. Storing a BLOB in a MySQL Database
12. Storing a BLOB in a PostgreSQL Database
13. Storing a BLOB in a SQLite Database
14. Generating a Dictionary Mapping Field Names to Column Numbers
15. Using dtuple for Flexible Accessto Query Results
16. Pretty-Printing the Contents of Database Cursors
17. Using a Single Parameter-Passing Style Across Various DB API Modules
18. Using Microsoft Jet via ADO
19. Accessing a JDBC Database from a Jython Servlet
20. Using ODBC to Get Excel Data with Jython
Chapter 8 Debugging and Testing
1. Introduction
2. Disabling Execution of Some Conditionals and Loops
3. Measuring Memory Usage on Linux
4. Debugging the Garbage-Collection Process
5. Trapping and Recording Exceptions
6. Tracing Expressions and Comments in Debug Mode
7. Getting More Information from Tracebacks
8. Starting the Debugger Automatically After an Uncaught Exception
9. Running Unit Tests Most Simply
10. Running Unit Tests Automatically
11. Using doctest with unittest in Python 2.4
12. Checking Values Against Intervals in Unit Testing
Chapter 9 Processes, Threads, and Synchronization
1. Introduction
2. Synchronizing All Methods in an Object
3. Terminating a Thread
4. Using a Queue.Queue as a Priority Queue
5. Working with a Thread Pool
6. Executing a Function in Parallel on Multiple Argument Sets
7. Coordinating Threads by Simple Message Passing
8. Storing Per-Thread Information
9. Multitasking Cooperatively Without Threads
10. Determining Whether Another Instanceof a Script Is Already Running in Windows
11. Processing Windows Messages Using MsgWaitForMultipleObjects
12. Driving an External Process with popen
13. Capturing the Output and Error Streams from a Unix Shell Command
14. Forking a Daemon Process on Unix
Chapter 10 System Administration
1. Introduction
2. Generating Random Passwords
3. Generating Easily Remembered Somewhat-Random Passwords
4. Authenticating Users by Means of a POP Server
5. Calculating Apache Hits per IP Address
6. Calculating the Rate of Client Cache Hits on Apache
7. Spawning an Editor from a Script
8. Backing Up Files
9. Selectively Copying a Mailbox File
10. Building a Whitelist of Email Addresses From a Mailbox
11. Blocking Duplicate Mails
12. Checking Your Windows Sound System
13. Registering or Unregistering a DLL on Windows
14. Checking and Modifying the Set of Tasks Windows Automatically Runs at Login
15. Creating a Share on Windows
16. Connecting to an Already Running Instance of Internet Explorer
17. Reading Microsoft Outlook Contacts
18. Gathering Detailed System Informationon Mac OS X
Chapter 11 User Interfaces
1. Introduction
2. Showing a Progress Indicator on a Text Console
3. Avoiding lambda in Writing Callback Functions
4. Using Default Values and Bounds with tkSimpleDialog Functions
5. Adding Drag and Drop Reordering to a Tkinter Listbox
6. Entering Accented Characters in Tkinter Widgets
7. Embedding Inline GIFs Using Tkinter
8. Converting Among Image Formats
9. Implementing a Stopwatch in Tkinter
10. Combining GUIs and Asynchronous I/Owith Threads
11. Using IDLE's Tree Widget in Tkinter
12. Supporting Multiple Values per Row in a Tkinter Listbox
13. Copying Geometry Methods and Options Between Tkinter Widgets
14. Implementing a Tabbed Notebook for Tkinter
15. Using a wxPython Notebook with Panels
16. Implementing an ImageJ Plug-in in Jython
17. Viewing an Image from a URL with Swing and Jython
18. Getting User Input on Mac OS
19. Building a Python Cocoa GUI Programmatically
20. Implementing Fade-in Windows with IronPython
Chapter 12 Processing XML
1. Introduction
2. Checking XML Well-Formedness
3. Counting Tags in a Document
4. Extracting Text from an XML Document
5. Autodetecting XML Encoding
6. Converting an XML Document into a Tree of Python Objects
7. Removing Whitespace-only Text Nodes from an XML DOM Node's Subtree
8. Parsing Microsoft Excel's XML
9. Validating XML Documents
10. Filtering Elements and Attributes Belonging to a Given Namespace
11. Merging Continuous Text Events with a SAX Filter
12. Using MSHTML to Parse XML or HTML
Chapter 13 Network Programming
1. Introduction
2. Passing Messages with Socket Datagrams
3. Grabbing a Document from the Web
4. Filtering a List of FTP Sites
5. Getting Time from a Server via the SNTP Protocol
6. Sending HTML Mail
7. Bundling Files in a MIME Message
8. Unpacking a Multipart MIME Message
9. Removing Attachments from an Email Message
10. Fixing Messages Parsed by Python 2.4 email.FeedParser
11. Inspecting a POP3 Mailbox Interactively
12. Detecting Inactive Computers
13. Monitoring a Network with HTTP
14. Forwarding and Redirecting Network Ports
15. Tunneling SSL Through a Proxy
16. Implementing the Dynamic IP Protocol
17. Connecting to IRC and Logging Messages to Disk
18. Accessing LDAP Servers
Chapter 14 Web Programming
1. Introduction
2. Testing Whether CGI Is Working
3. Handling URLs Within a CGI Script
4. Uploading Files with CGI
5. Checking for a Web Page's Existence
6. Checking Content Type via HTTP
7. Resuming the HTTP Download of a File
8. Handling Cookies While Fetching Web Pages
9. Authenticating with a Proxy for HTTPS Navigation
10. Running a Servlet with Jython
11. Finding an Internet Explorer Cookie
12. Generating OPML Files
13. Aggregating RSS Feeds
14. Turning Data into Web Pages Through Templates
15. Rendering Arbitrary Objects with Nevow
Chapter 15 Distributed Programming
1. Introduction
2. Making an XML-RPC Method Call
3. Serving XML-RPC Requests
4. Using XML-RPC with Medusa
5. Enabling an XML-RPC Server to Be Terminated Remotely
6. Implementing SimpleXMLRPCServer Niceties
7. Giving an XML-RPC Server a wxPython GUI
8. Using Twisted Perspective Broker
9. Implementing a CORBA Server and Client
10. Performing Remote Logins Using telnetlib
11. Performing Remote Logins with SSH
12. Authenticating an SSL Client over HTTPS
Chapter 16 Programs About Programs
1. Introduction
2. Verifying Whether a String Represents a Valid Number
3. Importing a Dynamically Generated Module
4. Importing from a Module Whose Name Is Determined at Runtime
5. Associating Parameters with a Function (Currying)
6. Composing Functions
7. Colorizing Python Source Using the Built-in Tokenizer
8. Merging and Splitting Tokens
9. Checking Whether a String Has Balanced Parentheses
10. Simulating Enumerations in Python
11. Referring to a List Comprehension While Building It
12. Automating the py2exe Compilation of Scripts into Windows Executables
13. Binding Main Script and Modules into One Executable on Unix
Chapter 17 Extending and Embedding
1. Introduction
2. Implementing a Simple Extension Type
3. Implementing a Simple Extension Type with Pyrex
4. Exposing a C++ Library to Python
5. Calling Functions from a Windows DLL
6. Using SWIG-Generated Modules in a Multithreaded Environment
7. Translating a Python Sequence into a C Array with the PySequence_Fast Protocol
8. Accessing a Python Sequence Item-by-Item with the Iterator Protocol
9. Returning None from a Python-Callable C Function
10. Debugging Dynamically Loaded C Extensions with gdb
11. Debugging Memory Problems
Chapter 18 Algorithms
1. Introduction
2. Removing Duplicates from a Sequence
3. Removing Duplicates from a Sequence While Maintaining Sequence Order
4. Generating Random Samples with Replacement
5. Generating Random Samples Without Replacement
6. Memoizing (Caching) the Return Values of Functions
7. Implementing a FIFO Container
8. Caching Objects with a FIFO Pruning Strategy
9. Implementing a Bag (Multiset) Collection Type
10. Simulating the Ternary Operator in Python
11. Computing Prime Numbers
12. Formatting Integers as Binary Strings
13. Formatting Integers as Strings in Arbitrary Bases
14. Converting Numbers to Rationals via Farey Fractions
15. Doing Arithmetic with Error Propagation
16. Summing Numbers with Maximal Accuracy
17. Simulating Floating Point
18. Computing the Convex Hulls and Diameters of 2D Point Sets
Chapter 19 Iterators and Generators
1. Introduction
2. Writing a range-like Function with Float Increments
3. Building a List from Any Iterable
4. Generating the Fibonacci Sequence
5. Unpacking a Few Items in a Multiple Assignment
6. Automatically Unpacking the Needed Number of Items
7. Dividing an Iterable into n Slices of Stride n
8. Looping on a Sequence by Overlapping Windows
9. Looping Through Multiple Iterables in Parallel
10. Looping Through the Cross-Product of Multiple Iterables
11. Reading a Text File by Paragraphs
12. Reading Lines with Continuation Characters
13. Iterating on a Stream of Data Blocks as a Stream of Lines
14. Fetching Large Record Sets from a Database with a Generator
15. Merging Sorted Sequences
16. Generating Permutations, Combinations, and Selections
17. Generating the Partitions of an Integer
18. Duplicating an Iterator
19. Looking Ahead into an Iterator
20. Simplifying Queue-Consumer Threads
21. Running an Iterator in Another Thread
22. Computing a Summary Report with itertools.groupby
Chapter 20 Descriptors, Decorators,and Metaclasses
1. Introduction
2. Getting Fresh Default Values at Each Function Call
3. Coding Properties as Nested Functions
4. Aliasing Attribute Values
5. Caching Attribute Values
6. Using One Method as Accessorfor Multiple Attributes
7. Adding Functionality to a Class by Wrapping a Method
8. Adding Functionality to a Class by Enriching All Methods
9. Adding a Method to a Class Instance at Runtime
10. Checking Whether Interfaces Are Implemented
11. Using _ _new_ _ and _ _init_ _ Appropriately in Custom Metaclasses
12. Allowing Chaining of Mutating List Methods
13. Using Cooperative Super calls with Terser Syntax
14. Initializing Instance Attributes Without Using _ _init_ _
15. Automatic Initialization of Instance Attributes
16. Upgrading Class Instances Automatically on reload
17. Binding Constants at Compile Time
18. Solving Metaclass Conflicts

Colophon

Return to Python Cookbook

Table of Contents

Chapter 1 Text

Introduction

Processing a String One Character at a Time

Converting Between Characters and Numeric Codes

Testing Whether an Object Is String-like

Aligning Strings

Trimming Space from the Ends of a String

Combining Strings

Reversing a String by Words or Characters

Checking Whether a String Contains a Set of Characters

Simplifying Usage of Strings' translate Method

Filtering a String for a Set of Characters

Checking Whether a String Is Text or Binary

Controlling Case

Accessing Substrings

Changing the Indentation of a Multiline String

Expanding and Compressing Tabs

Interpolating Variables in a String

Interpolating Variables in a Stringin Python 2.4

Replacing Multiple Patterns in a Single Pass

Checking a String for Any of Multiple Endings

Handling International Text with Unicode

Converting Between Unicode and Plain Strings

Printing Unicode Charactersto Standard Output

Encoding Unicode Data for XML and HTML

Making Some Strings Case-Insensitive

Converting HTML Documents to Texton a Unix Terminal

Chapter 2 Files

Introduction

Reading from a File

Writing to a File

Searching and Replacing Text in a File

Reading a Specific Line from a File

Counting Lines in a File

Processing Every Word in a File

Using Random-Access Input/Output

Updating a Random-Access File

Reading Data from zip Files

Handling a zip File Inside a String

Archiving a Tree of Files into a Compressed tar File

Sending Binary Data to Standard Output Under Windows

Using a C++-like iostream Syntax

Rewinding an Input File to the Beginning

Adapting a File-like Object to a True File Object

Walking Directory Trees

Swapping One File Extension for Another Throughout a Directory Tree

Finding a File Given a Search Path

Finding Files Given a Search Path and a Pattern

Finding a File on the Python Search Path

Dynamically Changing the PythonSearch Path

Computing the Relative Path from One Directory to Another

Reading an Unbuffered Character in a Cross-Platform Way

Counting Pages of PDF Documents on Mac OS X

Changing File Attributes on Windows

Extracting Text from OpenOffice.org Documents

Extracting Text from Microsoft Word Documents

File Locking Using a Cross-Platform API

Versioning Filenames

Calculating CRC-64 Cyclic Redundancy Checks

Chapter 3 Time and Money

Introduction

Calculating Yesterday and Tomorrow

Finding Last Friday

Calculating Time Periods in a Date Range

Summing Durations of Songs

Calculating the Number of Weekdays Between Two Dates

Looking up Holidays Automatically

Fuzzy Parsing of Dates

Checking Whether Daylight Saving Time Is Currently in Effect

Converting Time Zones

Running a Command Repeatedly

Scheduling Commands

Doing Decimal Arithmetic

Formatting Decimals as Currency

Using Python as a Simple Adding Machine

Checking a Credit Card Checksum

Watching Foreign Exchange Rates

Chapter 4 Python Shortcuts

Introduction