Python Cookbook, 3rd Edition

Table of contents

1. Preface
   1. Who This Book Is For
   2. Who This Book Is Not For
   3. Conventions Used in This Book
   4. Online Code Examples
   5. Using Code Examples
   6. Safari® Books Online
   7. How to Contact Us
   8. Acknowledgments
2. 1. Data Structures and Algorithms
   1. 1.1. Unpacking a Sequence into Separate Variables
   2. 1.2. Unpacking Elements from Iterables of Arbitrary Length
   3. 1.3. Keeping the Last N Items
   4. 1.4. Finding the Largest or Smallest N Items
   5. 1.5. Implementing a Priority Queue
   6. 1.6. Mapping Keys to Multiple Values in a Dictionary
   7. 1.7. Keeping Dictionaries in Order
   8. 1.8. Calculating with Dictionaries
   9. 1.9. Finding Commonalities in Two Dictionaries
   10. 1.10. Removing Duplicates from a Sequence while Maintaining Order
   11. 1.11. Naming a Slice
   12. 1.12. Determining the Most Frequently Occurring Items in a Sequence
   13. 1.13. Sorting a List of Dictionaries by a Common Key
   14. 1.14. Sorting Objects Without Native Comparison Support
   15. 1.15. Grouping Records Together Based on a Field
   16. 1.16. Filtering Sequence Elements
   17. 1.17. Extracting a Subset of a Dictionary
   18. 1.18. Mapping Names to Sequence Elements
   19. 1.19. Transforming and Reducing Data at the Same Time
   20. 1.20. Combining Multiple Mappings into a Single Mapping
3. 2. Strings and Text
   1. 2.1. Splitting Strings on Any of Multiple Delimiters
   2. 2.2. Matching Text at the Start or End of a String
   3. 2.3. Matching Strings Using Shell Wildcard Patterns
   4. 2.4. Matching and Searching for Text Patterns
   5. 2.5. Searching and Replacing Text
   6. 2.6. Searching and Replacing Case-Insensitive Text
   7. 2.7. Specifying a Regular Expression for the Shortest Match
   8. 2.8. Writing a Regular Expression for Multiline Patterns
   9. 2.9. Normalizing Unicode Text to a Standard Representation
   10. 2.10. Working with Unicode Characters in Regular Expressions
   11. 2.11. Stripping Unwanted Characters from Strings
   12. 2.12. Sanitizing and Cleaning Up Text
   13. 2.13. Aligning Text Strings
   14. 2.14. Combining and Concatenating Strings
   15. 2.15. Interpolating Variables in Strings
   16. 2.16. Reformatting Text to a Fixed Number of Columns
   17. 2.17. Handling HTML and XML Entities in Text
   18. 2.18. Tokenizing Text
   19. 2.19. Writing a Simple Recursive Descent Parser
   20. 2.20. Performing Text Operations on Byte Strings
4. 3. Numbers, Dates, and Times
   1. 3.1. Rounding Numerical Values
   2. 3.2. Performing Accurate Decimal Calculations
   3. 3.3. Formatting Numbers for Output
   4. 3.4. Working with Binary, Octal, and Hexadecimal Integers
   5. 3.5. Packing and Unpacking Large Integers from Bytes
   6. 3.6. Performing Complex-Valued Math
   7. 3.7. Working with Infinity and NaNs
   8. 3.8. Calculating with Fractions
   9. 3.9. Calculating with Large Numerical Arrays
   10. 3.10. Performing Matrix and Linear Algebra Calculations
   11. 3.11. Picking Things at Random
   12. 3.12. Converting Days to Seconds, and Other Basic Time Conversions
   13. 3.13. Determining Last Friday’s Date
   14. 3.14. Finding the Date Range for the Current Month
   15. 3.15. Converting Strings into Datetimes
   16. 3.16. Manipulating Dates Involving Time Zones
5. 4. Iterators and Generators
   1. 4.1. Manually Consuming an Iterator
   2. 4.2. Delegating Iteration
   3. 4.3. Creating New Iteration Patterns with Generators
   4. 4.4. Implementing the Iterator Protocol
   5. 4.5. Iterating in Reverse
   6. 4.6. Defining Generator Functions with Extra State
   7. 4.7. Taking a Slice of an Iterator
   8. 4.8. Skipping the First Part of an Iterable
   9. 4.9. Iterating Over All Possible Combinations or Permutations
   10. 4.10. Iterating Over the Index-Value Pairs of a Sequence
   11. 4.11. Iterating Over Multiple Sequences Simultaneously
   12. 4.12. Iterating on Items in Separate Containers
   13. 4.13. Creating Data Processing Pipelines
   14. 4.14. Flattening a Nested Sequence
   15. 4.15. Iterating in Sorted Order Over Merged Sorted Iterables
   16. 4.16. Replacing Infinite while Loops with an Iterator
6. 5. Files and I/O
   1. 5.1. Reading and Writing Text Data
   2. 5.2. Printing to a File
   3. 5.3. Printing with a Different Separator or Line Ending
   4. 5.4. Reading and Writing Binary Data
   5. 5.5. Writing to a File That Doesn’t Already Exist
   6. 5.6. Performing I/O Operations on a String
   7. 5.7. Reading and Writing Compressed Datafiles
   8. 5.8. Iterating Over Fixed-Sized Records
   9. 5.9. Reading Binary Data into a Mutable Buffer
   10. 5.10. Memory Mapping Binary Files
   11. 5.11. Manipulating Pathnames
   12. 5.12. Testing for the Existence of a File
   13. 5.13. Getting a Directory Listing
   14. 5.14. Bypassing Filename Encoding
   15. 5.15. Printing Bad Filenames
   16. 5.16. Adding or Changing the Encoding of an Already Open File
   17. 5.17. Writing Bytes to a Text File
   18. 5.18. Wrapping an Existing File Descriptor As a File Object
   19. 5.19. Making Temporary Files and Directories
   20. 5.20. Communicating with Serial Ports
   21. 5.21. Serializing Python Objects
7. 6. Data Encoding and Processing
   1. 6.1. Reading and Writing CSV Data
   2. 6.2. Reading and Writing JSON Data
   3. 6.3. Parsing Simple XML Data
   4. 6.4. Parsing Huge XML Files Incrementally
   5. 6.5. Turning a Dictionary into XML
   6. 6.6. Parsing, Modifying, and Rewriting XML
   7. 6.7. Parsing XML Documents with Namespaces
   8. 6.8. Interacting with a Relational Database
   9. 6.9. Decoding and Encoding Hexadecimal Digits
   10. 6.10. Decoding and Encoding Base64
   11. 6.11. Reading and Writing Binary Arrays of Structures
   12. 6.12. Reading Nested and Variable-Sized Binary Structures
   13. 6.13. Summarizing Data and Performing Statistics
8. 7. Functions
   1. 7.1. Writing Functions That Accept Any Number of Arguments
   2. 7.2. Writing Functions That Only Accept Keyword Arguments
   3. 7.3. Attaching Informational Metadata to Function Arguments
   4. 7.4. Returning Multiple Values from a Function
   5. 7.5. Defining Functions with Default Arguments
   6. 7.6. Defining Anonymous or Inline Functions
   7. 7.7. Capturing Variables in Anonymous Functions
   8. 7.8. Making an N-Argument Callable Work As a Callable with Fewer Arguments
   9. 7.9. Replacing Single Method Classes with Functions
   10. 7.10. Carrying Extra State with Callback Functions
   11. 7.11. Inlining Callback Functions
   12. 7.12. Accessing Variables Defined Inside a Closure
9. 8. Classes and Objects
   1. 8.1. Changing the String Representation of Instances
   2. 8.2. Customizing String Formatting
   3. 8.3. Making Objects Support the Context-Management Protocol
   4. 8.4. Saving Memory When Creating a Large Number of Instances
   5. 8.5. Encapsulating Names in a Class
   6. 8.6. Creating Managed Attributes
   7. 8.7. Calling a Method on a Parent Class
   8. 8.8. Extending a Property in a Subclass
   9. 8.9. Creating a New Kind of Class or Instance Attribute
   10. 8.10. Using Lazily Computed Properties
   11. 8.11. Simplifying the Initialization of Data Structures
   12. 8.12. Defining an Interface or Abstract Base Class
   13. 8.13. Implementing a Data Model or Type System
   14. 8.14. Implementing Custom Containers
   15. 8.15. Delegating Attribute Access
   16. 8.16. Defining More Than One Constructor in a Class
   17. 8.17. Creating an Instance Without Invoking init
   18. 8.18. Extending Classes with Mixins
   19. 8.19. Implementing Stateful Objects or State Machines
   20. 8.20. Calling a Method on an Object Given the Name As a String
   21. 8.21. Implementing the Visitor Pattern
   22. 8.22. Implementing the Visitor Pattern Without Recursion
   23. 8.23. Managing Memory in Cyclic Data Structures
   24. 8.24. Making Classes Support Comparison Operations
   25. 8.25. Creating Cached Instances
10. 9. Metaprogramming
    1. 9.1. Putting a Wrapper Around a Function
    2. 9.2. Preserving Function Metadata When Writing Decorators
    3. 9.3. Unwrapping a Decorator
    4. 9.4. Defining a Decorator That Takes Arguments
    5. 9.5. Defining a Decorator with User Adjustable Attributes
    6. 9.6. Defining a Decorator That Takes an Optional Argument
    7. 9.7. Enforcing Type Checking on a Function Using a Decorator
    8. 9.8. Defining Decorators As Part of a Class
    9. 9.9. Defining Decorators As Classes
    10. 9.10. Applying Decorators to Class and Static Methods
    11. 9.11. Writing Decorators That Add Arguments to Wrapped Functions
    12. 9.12. Using Decorators to Patch Class Definitions
    13. 9.13. Using a Metaclass to Control Instance Creation
    14. 9.14. Capturing Class Attribute Definition Order
    15. 9.15. Defining a Metaclass That Takes Optional Arguments
    16. 9.16. Enforcing an Argument Signature on *args and **kwargs
    17. 9.17. Enforcing Coding Conventions in Classes
    18. 9.18. Defining Classes Programmatically
    19. 9.19. Initializing Class Members at Definition Time
    20. 9.20. Implementing Multiple Dispatch with Function Annotations
    21. 9.21. Avoiding Repetitive Property Methods
    22. 9.22. Defining Context Managers the Easy Way
    23. 9.23. Executing Code with Local Side Effects
    24. 9.24. Parsing and Analyzing Python Source
    25. 9.25. Disassembling Python Byte Code
11. 10. Modules and Packages
    1. 10.1. Making a Hierarchical Package of Modules
    2. 10.2. Controlling the Import of Everything
    3. 10.3. Importing Package Submodules Using Relative Names
    4. 10.4. Splitting a Module into Multiple Files
    5. 10.5. Making Separate Directories of Code Import Under a Common Namespace
    6. 10.6. Reloading Modules
    7. 10.7. Making a Directory or Zip File Runnable As a Main Script
    8. 10.8. Reading Datafiles Within a Package
    9. 10.9. Adding Directories to sys.path
    10. 10.10. Importing Modules Using a Name Given in a String
    11. 10.11. Loading Modules from a Remote Machine Using Import Hooks
    12. 10.12. Patching Modules on Import
    13. 10.13. Installing Packages Just for Yourself
    14. 10.14. Creating a New Python Environment
    15. 10.15. Distributing Packages
12. 11. Network and Web Programming
    1. 11.1. Interacting with HTTP Services As a Client
    2. 11.2. Creating a TCP Server
    3. 11.3. Creating a UDP Server
    4. 11.4. Generating a Range of IP Addresses from a CIDR Address
    5. 11.5. Creating a Simple REST-Based Interface
    6. 11.6. Implementing a Simple Remote Procedure Call with XML-RPC
    7. 11.7. Communicating Simply Between Interpreters
    8. 11.8. Implementing Remote Procedure Calls
    9. 11.9. Authenticating Clients Simply
    10. 11.10. Adding SSL to Network Services
    11. 11.11. Passing a Socket File Descriptor Between Processes
    12. 11.12. Understanding Event-Driven I/O
    13. 11.13. Sending and Receiving Large Arrays
13. 12. Concurrency
    1. 12.1. Starting and Stopping Threads
    2. 12.2. Determining If a Thread Has Started
    3. 12.3. Communicating Between Threads
    4. 12.4. Locking Critical Sections
    5. 12.5. Locking with Deadlock Avoidance
    6. 12.6. Storing Thread-Specific State
    7. 12.7. Creating a Thread Pool
    8. 12.8. Performing Simple Parallel Programming
    9. 12.9. Dealing with the GIL (and How to Stop Worrying About It)
    10. 12.10. Defining an Actor Task
    11. 12.11. Implementing Publish/Subscribe Messaging
    12. 12.12. Using Generators As an Alternative to Threads
    13. 12.13. Polling Multiple Thread Queues
    14. 12.14. Launching a Daemon Process on Unix
14. 13. Utility Scripting and System Administration
    1. 13.1. Accepting Script Input via Redirection, Pipes, or Input Files
    2. 13.2. Terminating a Program with an Error Message
    3. 13.3. Parsing Command-Line Options
    4. 13.4. Prompting for a Password at Runtime
    5. 13.5. Getting the Terminal Size
    6. 13.6. Executing an External Command and Getting Its Output
    7. 13.7. Copying or Moving Files and Directories
    8. 13.8. Creating and Unpacking Archives
    9. 13.9. Finding Files by Name
    10. 13.10. Reading Configuration Files
    11. 13.11. Adding Logging to Simple Scripts
    12. 13.12. Adding Logging to Libraries
    13. 13.13. Making a Stopwatch Timer
    14. 13.14. Putting Limits on Memory and CPU Usage
    15. 13.15. Launching a Web Browser
15. 14. Testing, Debugging, and Exceptions
    1. 14.1. Testing Output Sent to stdout
    2. 14.2. Patching Objects in Unit Tests
    3. 14.3. Testing for Exceptional Conditions in Unit Tests
    4. 14.4. Logging Test Output to a File
    5. 14.5. Skipping or Anticipating Test Failures
    6. 14.6. Handling Multiple Exceptions
    7. 14.7. Catching All Exceptions
    8. 14.8. Creating Custom Exceptions
    9. 14.9. Raising an Exception in Response to Another Exception
    10. 14.10. Reraising the Last Exception
    11. 14.11. Issuing Warning Messages
    12. 14.12. Debugging Basic Program Crashes
    13. 14.13. Profiling and Timing Your Program
    14. 14.14. Making Your Programs Run Faster
16. 15. C Extensions
    1. 15.1. Accessing C Code Using ctypes
    2. 15.2. Writing a Simple C Extension Module
    3. 15.3. Writing an Extension Function That Operates on Arrays
    4. 15.4. Managing Opaque Pointers in C Extension Modules
    5. 15.5. Defining and Exporting C APIs from Extension Modules
    6. 15.6. Calling Python from C
    7. 15.7. Releasing the GIL in C Extensions
    8. 15.8. Mixing Threads from C and Python
    9. 15.9. Wrapping C Code with Swig
    10. 15.10. Wrapping Existing C Code with Cython
    11. 15.11. Using Cython to Write High-Performance Array Operations
    12. 15.12. Turning a Function Pointer into a Callable
    13. 15.13. Passing NULL-Terminated Strings to C Libraries
    14. 15.14. Passing Unicode Strings to C Libraries
    15. 15.15. Converting C Strings to Python
    16. 15.16. Working with C Strings of Dubious Encoding
    17. 15.17. Passing Filenames to C Extensions
    18. 15.18. Passing Open Files to C Extensions
    19. 15.19. Reading File-Like Objects from C
    20. 15.20. Consuming an Iterable from C
    21. 15.21. Diagnosing Segmentation Faults
17. A. Further Reading
    1. Online Resources
    2. Books for Learning Python
    3. Advanced Books
18. Index
19. Colophon
20. Copyright