Programming Python | O'Reilly Media

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

This book is about using Python, an easy-to-use, flexible, object-oriented, mature, popular, and open source programming language designed to optimize development speed. Although it is completely general purpose, Python is often called a scripting language, partly because of its sheer ease of use and partly because it is commonly used to orchestrate or "glue" other software components in an application. Python is also commonly known as a high-level language, because it automates most low-level tasks that programmers must handle manually in traditional languages such as C.

If you are new to Python, chances are you've heard about the language somewhere but are not quite sure what it is about. To help you get started, this chapter provides a general introduction to Python's features and roles. Most of it will make more sense once you have seen real Python programs, but let's first take a quick pass over the forest before wandering among the trees. In this chapter, we'll explore Python's philosophy, its history, and some of its most prominent benefits and uses, before digging into the details.

In the Preface, I mentioned that Python emphasizes concepts such as quality, productivity, portability, and integration. Since these four terms summarize most of the reasons for using Python, I'd like to define them in a bit more detail.

Software quality: Python makes it easy to write software that can be understood, reused, and modified. It was deliberately designed to raise development quality expectations in the scripting world. Python's clear syntax and coherent design, for example, almost force programmers to write readable code—a critical feature for software that may be changed or reused by others in the future.
Of equal importance, because the Python language tries to do better, so too do Python developers and the Python community at large. In the Python world, one finds a refreshing focus on quality concepts such as simplicity, explicitness, and readability—ideas often given little more than a passing glance in some camps. (For more on this Python-inspired mindset, see the sidebar "The Python 'Secret Handshake'," near the end of this chapter.)

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

This book is about using Python, an easy-to-use, flexible, object-oriented, mature, popular, and open source programming language designed to optimize development speed. Although it is completely general purpose, Python is often called a scripting language, partly because of its sheer ease of use and partly because it is commonly used to orchestrate or "glue" other software components in an application. Python is also commonly known as a high-level language, because it automates most low-level tasks that programmers must handle manually in traditional languages such as C.

If you are new to Python, chances are you've heard about the language somewhere but are not quite sure what it is about. To help you get started, this chapter provides a general introduction to Python's features and roles. Most of it will make more sense once you have seen real Python programs, but let's first take a quick pass over the forest before wandering among the trees. In this chapter, we'll explore Python's philosophy, its history, and some of its most prominent benefits and uses, before digging into the details.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

In the Preface, I mentioned that Python emphasizes concepts such as quality, productivity, portability, and integration. Since these four terms summarize most of the reasons for using Python, I'd like to define them in a bit more detail.

Software quality: Python makes it easy to write software that can be understood, reused, and modified. It was deliberately designed to raise development quality expectations in the scripting world. Python's clear syntax and coherent design, for example, almost force programmers to write readable code—a critical feature for software that may be changed or reused by others in the future.
Of equal importance, because the Python language tries to do better, so too do Python developers and the Python community at large. In the Python world, one finds a refreshing focus on quality concepts such as simplicity, explicitness, and readability—ideas often given little more than a passing glance in some camps. (For more on this Python-inspired mindset, see the sidebar "The Python 'Secret Handshake'," near the end of this chapter.)
The Python language really does look like it was designed and not accumulated. It has an orthogonal, explicit, and minimalist design that makes code easy to understand and easy to predict. Python approaches complexity by providing a simple core language and splitting application-specific tools into a large set of modular library components.
As a popular slogan attests, the result is that Python "fits your brain"—it's possible to use the language without constantly flipping through reference manuals. This design makes Python ideal as a customization language for nonexperts. Perhaps most important is that by limiting the number of possible interactions in your code, Python reduces both program complexity and the potential for bugs.
Besides being well designed, Python is also well tooled for modern software methodologies such as structured, modular, and object-oriented design, which allow code to be written once and reused many times. In fact, due to the inherent power and flexibility of the language, writing high-quality Python components that may be applied in multiple contexts is almost automatic.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Python was invented around 1990 by Guido van Rossum, when he was at CWI in Amsterdam. It is named after the BBC comedy series Monty Python 's Flying Circus, of which Guido is a fan (see this chapter's sidebar "What's in a Name?"). Guido was also involved with the Amoeba distributed operating system and the ABC language. In fact, his original motivation for creating Python was to create an advanced scripting language for the Amoeba system. Moreover, Python borrowed many of the usability-study-inspired ideas in ABC, but added practicality in the form of libraries, datatypes, external interfaces, and more.

The net effect was that Python's design turned out to be general enough to address a wide variety of domains. It is now used in increasingly diverse roles by hundreds of thousands of engineers around the world. Companies use Python today in commercial products for tasks as diverse as web site construction, hardware testing, numeric analysis, customizing C++ and Java class libraries, movie animation, and much more (more on roles in the next section). In fact, because Python is a completely general-purpose language, its target domains are limited only by the scope of computers in general.

Since it first appeared on the public domain scene in 1991, Python has continued to attract a loyal following and has spawned a dedicated Internet newsgroup, comp.lang.python, in 1994. As the first edition of this book was being written in 1995, Python's home page debuted on the Web at http://www.python.org—still the official place to find all things Python. A supplemental site, the Vaults of Parnassus, serves as a library of third-party extensions for Python application development (see http://www.vex.net/parnassus). More recently, the Python Package Index site (PyPI at http://www.python.org/pypi—also known as the "Python Cheese Shop"—began providing a comprehensive and automated catalog of third-party Python packages.

To help manage Python's growth, organizations that are aimed at supporting Python developers have taken shape over the years: among them, the now defunct Python Software Activity (PSA) was formed to help facilitate Python conferences and web sites, and the Python Consortium was formed by organizations interested in helping to foster Python's growth. More recently, the Python Software Foundation (PSF) was formed to own the intellectual property of Python and coordinate community activities, and the Python Business Forum (PBF) nonprofit group addresses the needs of companies whose businesses are based on Python. Additional resources are available for Python training, consulting, and other services.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

It's been an exciting decade in the Python world. Since I wrote the first edition of this book in 1995 and 1996, Python has grown from a new kid on the scripting-languages block to an established and widely used tool in companies around the world. In fact, today the real question is not who is using Python, but who is not. Python is now used in some fashion in almost every software organization—whether as a tactical tool for quick tasks or an implementation language for longer-range strategic projects.

Although measuring the popularity of an open source, freely distributed tool such as Python is not always easy (there are no licenses to be tallied), most available statistics reveal exponential growth in Python's popularity over the last decade. Among the most recent signs of Python's explosive growth are:

Users

In 1999, one leading industry observer suggested that, based on various statistics, there were as many as 300,000 Python users worldwide. Other estimates are still more optimistic. In early 2000, for instance, the Python web site was already on track to service 500,000 new Python interpreter downloads by year end in addition to other Python distribution media. Python is also a standard preinstalled item on Linux, Macintosh, and some Windows computers today and is embedded in various applications and hardware.

Today, the best estimates, based on developer surveys and network activity, suggest that there are likely between 750,000 and 1 million Python users worldwide. A better estimate is impossible because of Python's open source nature, but Python clearly enjoys a large and active user community.

Applications

Real organizations have adopted Python and Python-focused systems for real projects. It has been used to:

Animate movies (Industrial Light & Magic, Sony Pictures Imageworks, Disney, Pixar)
Perform searches on the Internet (Google, Infoseek)

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

One way to describe a language is by listing its features. Of course, this will be more meaningful after you've seen Python in action; the best I can do now is speak in the abstract. And it's really how Python's features work together that make it what it is. But looking at some of Python's attributes may help define it; Table 1-1 lists some of the common reasons cited for Python's appeal.

Table 1-1: Python language features
Features	Benefits
No manual compile or link steps	Rapid development cycle turnaround
No type declarations	Simpler, shorter, and more flexible programs
Automatic memory management	Garbage collection avoids bookkeeping code and errors
High-level datatypes and operations	Fast development using built-in object types
Object-oriented programming	Code reuse; C++, Java, COM, and .NET integration
Embedding and extending in C	Optimization, customization, legacy code, system "glue"
Classes, modules, exceptions	Modular "programming-in-the-large" support for large-scale projects
A simple, clear syntax and design	Readability, maintainability, ease of learning, less potential for bugs

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Because Python is used in a wide variety of ways, it's almost impossible to give an authoritative answer to this question. As a general-purpose language, Python can be used for almost anything computers are capable of. Its feature set applies to both rapid and longer-term development modes. And from an abstract perspective, any project that can benefit from the inclusion of a language optimized for speed of development is a good target Python application domain. Given the ever-shrinking schedules in software development, this is a very broad category.

A more specific answer is less easy to formulate. For instance, some use Python as an embedded extension language, and others use it exclusively as a standalone programming tool. To some extent, this entire book will answer this very question—it explores some of Python's most common roles. For now, here's a summary of some of the more common ways Python is being applied today:

System utilities: Portable command-line tools, testing, system administration scripts
Internet scripting: CGI web sites, Java applets, XML, email, Zope/Plone, CherryPy, Webware, Twisted
GUIs: With tools such as Tk, wxPython, Qt, Gtk, PythonCard, Dabo, Swing, Anygui
Component integration: C/C++ library frontends, product customization
Database access: Persistent object stores, SQL database interfaces
Distributed programming: With client/server APIs like CORBA, CGI, COM, .NET, SOAP, XML-RPC
Rapid-prototyping/development: Tactical run-once programs or deliverable prototypes
Language-based modules: Replacing special-purpose parsers with Python
And more: Image processing, numeric programming, gaming, AI, etc.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

To be fair again, some tasks are outside of Python's scope. Like all dynamic interpreted languages, Python, as currently implemented, isn't generally as fast or efficient as static, compiled languages such as C (see the earlier sidebar, "How Python Runs Your Code," for the technical story). At least when nontypical benchmarks are compared line for line, Python code runs more slowly than C code.

Whether you will ever care about this difference in execution speed depends upon the sorts of applications you will write. In many domains, the difference doesn't matter at all; for programs that spend most of their time interacting with users or transferring data over networks, Python is usually more than adequate to meet the performance needs of the entire application by itself.

Moreover, most realistic Python programs tend to run very near the speed of the C language anyhow. Because system interactions such as accessing files or creating GUIs are implemented by linked-in C language code in the standard implementation, typical Python programs are often nearly as fast as equivalent C language programs. In fact, because Python programs use highly optimized data structures and libraries, they are sometimes quicker than C programs that must implement such tools manually.

In some domains, however, efficiency is still a main priority. Programs that spend most of their time in intense number crunching, for example, will usually be slower in Python than in fully compiled languages. Because it is interpreted today, Python alone usually isn't the best tool for the delivery of such performance- critical components. Instead, computationally intensive operations can be implemented as compiled extensions to Python and coded in a low-level language such as C. Python can't be used as the sole implementation language for such components, but it works well as a frontend scripting interface to them.

For example, numerical programming and image processing support has been added to Python by combining optimized extensions with a Python language interface. In such a system, once the optimized extensions have been developed, most of the programming occurs at the simpler level of Python scripting. The net result is a numerical programming tool that's both efficient and easy to use. The NumPy extension (and its NumArray and ScientificPython relatives), for instance, adds vector processing to Python, turning it into what has been called an open source equivalent to Matlab.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

In this book's conclusion—after we've had a chance to study Python in action—we will return to some of the bigger ideas introduced in this chapter. I want to point out up front, though, that my background is in computer science, not marketing. I plan to be brutally honest in this book, both about Python's features and about its downsides. Despite the fact that Python is one of the most easy-to-use and flexible programming languages ever created, there are indeed some pitfalls, which we will not gloss over in this book.

Let's start now. One of the first pitfalls you should know about, and a common remark made by Python newcomers, is this: Python makes it incredibly easy to quickly throw together a bad design. For some, it seems a genuine problem. Because developing programs in Python is so simple and fast compared with using traditional languages, it's easy to get wrapped up in the act of programming itself and pay less attention to the problem you are really trying to solve. If you haven't done any Python development yet, you'll find that it is an incremental, interactive, and rapid experience that encourages experimentation.

In fact, Python can be downright seductive—so much so that you may need to consciously resist the temptation to quickly implement a program in Python that works, is loaded with features, and is arguably "cool," but that leaves you as far from a maintainable implementation of your original conception as you were when you started. The natural delays built into compiled language development—fixing compiler error messages, linking libraries, and the like—aren't there in Python to apply the brakes. In fact, it's not uncommon for a Python program to run the first time you try it; there is much less syntax and there are far fewer procedures to get in your way.

This isn't necessarily all bad, of course. In most cases, the early designs that you throw together fast are steppingstones to better designs that you later keep. That is the nature of prototyping, after all, and often the reality of programming under tight schedules. But you should be warned: even with a rapid development language such as Python, there is no substitute for brains—it's always best to think before you start typing code. To date, at least, no computer programming language has managed to make "wetware" obsolete.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

If you are like most people, when you pick up a book as large as this one, you'd like to know a little about what you're going to be learning before you roll up your sleeves. That's what this chapter is for—it provides a demonstration of some of the kinds of things you can do with Python, before getting into the details. You won't learn much here, and if you're looking for explanations of the tools and techniques applied in this chapter, you'll have to read on to later parts of the book. The point here is just to whet your appetite, review a few Python basics, and preview some of the topics to come.

To do this, I'll pick a fairly simple application task—constructing a database of records—and migrate it through multiple steps: interactive coding, command-line tools, console interfaces, GUIs, and simple web-based interfaces. Along the way, we'll also peek at concepts such as data representation, object persistence, and object-oriented programming (OOP); I'll mention some alternatives that we'll revisit later in the book; and I'll review some core Python ideas that you should be aware of before reading this book. Ultimately, we'll wind up with a database of Python class instances, which can be browsed and changed from a variety of interfaces.

I'll cover additional topics in this book, of course, but the techniques you will see here are representative of some of the domains we'll explore later. And again, if you don't completely understand the programs in this chapter, don't worry because you shouldn't—not yet anyway. This is just a Python demo. We'll fill in the details soon enough. For now, let's start off with a bit of fun.

Imagine, if you will, that you need to keep track of information about people for some reason; maybe you want to store an address book on your computer, or perhaps you need to keep track of employees in a small business. For whatever reason, you want to write a program that keeps track of details about these people. In other words, you want to keep records in a database—to permanently store lists of people's attributes on your computer.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

If you are like most people, when you pick up a book as large as this one, you'd like to know a little about what you're going to be learning before you roll up your sleeves. That's what this chapter is for—it provides a demonstration of some of the kinds of things you can do with Python, before getting into the details. You won't learn much here, and if you're looking for explanations of the tools and techniques applied in this chapter, you'll have to read on to later parts of the book. The point here is just to whet your appetite, review a few Python basics, and preview some of the topics to come.

To do this, I'll pick a fairly simple application task—constructing a database of records—and migrate it through multiple steps: interactive coding, command-line tools, console interfaces, GUIs, and simple web-based interfaces. Along the way, we'll also peek at concepts such as data representation, object persistence, and object-oriented programming (OOP); I'll mention some alternatives that we'll revisit later in the book; and I'll review some core Python ideas that you should be aware of before reading this book. Ultimately, we'll wind up with a database of Python class instances, which can be browsed and changed from a variety of interfaces.

I'll cover additional topics in this book, of course, but the techniques you will see here are representative of some of the domains we'll explore later. And again, if you don't completely understand the programs in this chapter, don't worry because you shouldn't—not yet anyway. This is just a Python demo. We'll fill in the details soon enough. For now, let's start off with a bit of fun.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Imagine, if you will, that you need to keep track of information about people for some reason; maybe you want to store an address book on your computer, or perhaps you need to keep track of employees in a small business. For whatever reason, you want to write a program that keeps track of details about these people. In other words, you want to keep records in a database—to permanently store lists of people's attributes on your computer.

Naturally, there are off-the-shelf programs for managing databases like these. By writing a program for this task yourself, however, you'll have complete control over its operation; you can add code for special cases and behaviors that precoded software may not have anticipated. You won't have to install and learn to use yet another database product. And you won't be at the mercy of a software vendor to fix bugs or add new features. You decide to write a Python program to manage your people.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

If we're going to store records in a database, the first step is probably deciding what those records will look like. There are a variety of ways to represent information about people in the Python language. Built-in object types such as lists and dictionaries are often sufficient, especially if we don't care about processing the data we store.

Lists, for example, can collect attributes about people in a positionally ordered way. Start up your Python interactive interpreter and type the following two statements (this works in the IDLE GUI, after typing python at a shell prompt, and so on, and the >>> characters are Python's prompt—if you've never run Python code this way before, see an introductory resource such as O'Reilly's Learning Python for help with getting started):

>>>bob = ['Bob Smith', 42, 30000, 'software']
>>> sue = ['Sue Jones', 45, 40000, 'music']

We've just made two records, albeit simple ones, to represent two people, Bob and Sue (my apologies if you really are Bob or Sue, generically or otherwise). Each record is a list of four properties: name, age, pay, and job field. To access these fields, we simply index by position (the result is in parentheses here because it is a tuple of two results):

>>>bob[0], sue[2]             # fetch name, pay
('Bob Smith', 40000)

Processing records is easy with this representation; we just use list operations. For example, we can extract a last name by splitting the name field on blanks and grabbing the last part, and we may give someone a raise by changing their list in-place:

>>>bob[0].split( )[-1]         # what's bob's last name?
'Smith'
>>> sue[2] *= 1.25             # give sue a 25% raise
>>> sue
['Sue Jones', 45, 50000.0, 'music']

The last-name expression here proceeds from left to right: we fetch Bob's name, split it into a list of substrings around spaces, and index his last name (run it one step at a time to see how).

Section 2.3.1.1: A database list

Of course, what we really have at this point is just two variables, not a database; to collect Bob and Sue into a unit, we might simply stuff them into another list:

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

So far, we've settled on a dictionary-based representation for our database of records, and we've reviewed some Python data structure concepts along the way. As mentioned, though, the objects we've seen so far are temporary—they live in memory and they go away as soon as we exit Python or the Python program that created them. To make our people persistent, they need to be stored in a file of some sort.

One way to keep our data around between program runs is to write all the data out to a simple text file, in a formatted way. Provided the saving and loading tools agree on the format selected, we're free to use any custom scheme we like.

Section 2.4.1.1: Test data script

So that we don't have to keep working interactively, let's first write a script that initializes the data we are going to store (if you've done any Python work in the past, you know that the interactive prompt tends to become tedious once you leave the realm of simple one-liners). Example 2-1 creates the sort of records and database dictionary we've been working with so far, but because it is a module, we can import it repeatedly without having to retype the code each time. In a sense, this module is a database itself, but its program code format doesn't support automatic or end-user updates as is.

Besides allowing us to associate meaningful labels with data rather than numeric positions, dictionaries are often more flexible than lists, especially when there isn't a fixed size to our problem. For instance, suppose you need to sum up columns of data stored in a text file where the number of columns is not known or fixed:

>>>print open('data.txt').read( )
001.1 002.2 003.3
010.1 020.2 030.3 040.4
100.1 200.2 300.3

Here, we cannot preallocate a fixed-length list of sums because the number of columns may vary. Splitting on whitespace extracts the columns, and float converts to numbers, but a fixed-size list won't easily accommodate a set of sums (at least, not without extra code to manage its size). Dictionaries are more convenient here because we can use column positions as keys instead of using absolute offsets Most of this code uses tools added to Python in the last five years; see Chapter 4 for more on file iterators, Chapter 21 for text processing and alternative summers, and the library manual for the 2.3

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Let's step back for a moment and consider how far we've come. At this point, we've created a database of records: the shelve, as well as per-record pickle file approaches of the prior section suffice for basic data storage tasks. As is, our records are represented as simple dictionaries, which provide easier-to-understand access to fields than do lists (by key, rather than by position). Dictionaries, however, still have some limitations that may become more critical as our program grows over time.

For one thing, there is no central place for us to collect record processing logic. Extracting last names and giving raises, for instance, can be accomplished with code like the following:

>>>import shelve
>>> db = shelve.open('people-shelve')
>>> bob = db['bob']
>>> bob['name'].split( )[-1]             # get bob's last name
'Smith'
>>> sue = db['sue']
>>> sue['pay'] *= 1.25                  # give sue a raise
>>> sue['pay']
75000.0
>>> db['sue'] = sue
>>> db.close( )

This works, and it might suffice for some short programs. But if we ever need to change the way last names and raises are implemented, we might have to update this kind of code in many places in our program. In fact, even finding all such magical code snippets could be a challenge; hardcoding or cutting and pasting bits of logic redundantly like this in more than one place will almost always come back to haunt you eventually.

It would be better to somehow hide—that is, encapsulate—such bits of code. Functions in a module would allow us to implement such operations in a single place and thus avoid code redundancy, but still wouldn't naturally associate them with the records themselves. What we'd like is a way to bind processing logic with the data stored in the database in order to make it easier to understand, debug, and reuse.

Another downside to using dictionaries for records is that they are difficult to expand over time. For example, suppose that the set of data fields or the procedure for giving raises is different for different kinds of people (perhaps some people get a bonus each year and some do not). If we ever need to extend our program, there is no natural way to customize simple dictionaries. For future growth, we'd also like our software to support extension and customization in a natural way.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

So far, our database program consists of class instances stored in a shelve file, as coded in the preceding section. It's sufficient as a storage medium, but it requires us to run scripts from the command line or type code interactively in order to view or process its content. Improving on this is straightforward: simply code more general programs that interact with users, either from a console window or from a full-blown graphical interface.

Let's start with something simple. The most basic kind of interface we can code would allow users to type keys and values in a console window in order to process the database (instead of writing Python program code). Example 2-21, for instance, implements a simple interactive loop that allows a user to query multiple record objects in the shelve by key.

Example 2-21. PP3E\Preview\peopleinteract_query.py

# interactive queries
import shelve
fieldnames = ('name', 'age', 'job', 'pay')
maxfield   = max(len(f) for f in fieldnames)
db = shelve.open('class-shelve')

while True:
    key = raw_input('\nKey? => ')       # key or empty line, exc at eof
    if not key: break
    try:
        record = db[key]                # fetch by key, show in console
    except:
        print 'No such key "%s"!' % key
    else:
        for field in fieldnames:
            print field.ljust(maxfield), '=>', getattr(record, field)

This script uses getattr to fetch an object's attribute when given its name string, and the ljust left-justify method of strings to align outputs (maxfield, derived from a comprehension expression, is the length of the longest field name). When run, this script goes into a loop, inputting keys from the interactive user (technically, from the standard input stream, which is usually a console window) and displaying the fetched records field by field. An empty line ends the session:

Key? =>sue
name => Sue Jones
age  => 45
job  => music
pay  => 40000

Key? => nobody
No such key "nobody"!

Key? =>

Example 2-22 goes further and allows interactive updates. For an input key, it inputs values for each field and either updates an existing record or creates a new object and stores it under the key.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The console-based interface approach of the preceding section works, and it may be sufficient for some users assuming that they are comfortable with typing commands in a console window. With just a little extra work, though, we can add a GUI that is more modern, easier to use and less error prone, and arguably sexier.

As we'll see later in this book, a variety of GUI toolkits and builders are available for Python programmers: Tkinter, wxPython, PyQt, PythonCard, Dabo, and more. Of these, Tkinter ships with Python, and it is something of a de facto standard.

Tkinter is a lightweight toolkit and so meshes well with a scripting language such as Python; it's easy to do basic things with Tkinter, and it's straightforward to do more advanced things with extensions and OOP-based code. As an added bonus, Tkinter GUIs are portable across Windows, Linux/Unix, and Macintosh; simply copy the source code to the machine on which you wish to use your GUI.

Because Tkinter is designed for scripting, coding GUIs with it is straightforward. We'll study all of its concepts and tools later in this book. But as a first example, the first program in Tkinter is just a few lines of code, as shown in Example 2-23.

Example 2-23. PP3E\Preview\tkinter001.py

from Tkinter import *
Label(text='Spam').pack( )
mainloop( )

This isn't the most useful GUI ever coded, but it demonstrates Tkinter basics and it builds the fully functional window shown in Figure 2-1 in just three simple lines of code. From the Tkinter module, we get widget (screen device) construction calls such as Label, geometry manager methods such as pack, widget configuration constants such as TOP and RIGHT side hints for pack, and the mainloop call, which starts event processing.

Figure 2-1: tkinter001.py window

You can launch this example in IDLE from a console command line by clicking its icon the same way you can run other Python scripts. Tkinter itself is a standard part of Python and works out-of-the-box on Windows, though you may need to install extras on some computers (more details later in this book).

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

GUI interfaces are easier to use than command lines and are often all we need to simplify access to data. By making our database available on the Web, we can open it up to even wider use. Anyone with Internet access and a web browser can access the data, regardless of where they are located and which machine they are using. Anything from workstations to cell phones will suffice. Moreover, web-based interfaces require only a web browser; there is no need to install Python to access the data except on the single-server machine. Although web-based approaches may sacrifice some of the utility and speed of in-process GUI toolkits, their portability gain can be compelling.

As we'll also see later in this book, there are a variety of ways to go about scripting interactive web pages of the sort we'll need in order to access our data. Basic CGI scripting is more than adequate for simple tasks like ours. For more advanced applications, toolkits and frameworks such as Zope, Plone, Twisted, CherryPy, Webware, Django, TurboGears, mod_python, and Quixote can provide tools that we would otherwise need to code from scratch. Zope, for instance, simplifies many CGI scripting tasks and provides for security, load balancing on the server, and more. For now, let's keep things simple and code a CGI script.

CGI scripting in Python is easy as long as you already have a handle on things like HTML forms, URLs, and the client/server model of the Web (all topics we'll address in detail later in this book). Whether you're aware of all the underlying details or not, the basic interaction model is probably familiar.

In a nutshell, a user visits a web site and receives a form, coded in HTML, to be filled out in her browser. After submitting the form, a script, identified within either the form or the address used to contact the server, is run on the server and produces another HTML page as a reply. Along the way, data typically passes through three programs: from the client browser, to the web server, to the CGI script, and back again to the browser. This is a natural model for the database access interaction we're after—users can submit a database key to the server and receive the corresponding record as a reply page.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

And that concludes our sneak preview demo of Python in action. We've explored data representation, OOP, object persistence, GUIs, and web site basics. We haven't studied any of these topics in any sort of depth. Hopefully, though, this chapter has piqued your curiosity about Python applications programming.

In the rest of this book, we'll delve into these and other application programming tools and topics, in order to help you put Python to work in your own programs. In the next chapter, we begin our tour with the systems programming tools available to Python programmers.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

This chapter begins our in-depth look at ways to apply Python to real programming tasks. In this and the following chapters, you'll see how to use Python to write system tools, GUIs, database applications, Internet scripts, web sites, and more. Along the way, we'll also study larger Python programming concepts in action: code reuse, maintainability, object-oriented programming (OOP), and so on.

In this first part of the book, we begin our Python programming tour by exploring the systems application domain— scripts that deal with files, programs, and the general environment surrounding a program. Although the examples in this domain focus on particular kinds of tasks, the techniques they employ will prove to be useful in later parts of the book as well. In other words, you should begin your journey here, unless you are already a Python systems programming wizard.

Python's system interfaces span application domains, but for the next five chapters, most of our examples fall into the category of system tools—programs sometimes called command-line utilities, shell scripts, and other permutations of such words. Regardless of their title, you are probably already familiar with this sort of script; these scripts accomplish such tasks as processing files in a directory, launching test scripts, and so on. Such programs historically have been written in nonportable and syntactically obscure shell languages such as DOS batch files, csh, and awk.

Even in this relatively simple domain, though, some of Python's better attributes shine brightly. For instance, Python's ease of use and extensive built-in library make it simple (and even fun) to use advanced system tools such as threads, signals, forks, sockets, and their kin; such tools are much less accessible under the obscure syntax of shell languages and the slow development cycles of compiled languages. Python's support for concepts like code clarity and OOP also help us write shell tools that can be read, maintained, and reused. When using Python, there is no need to start every new script from scratch.

Moreover, we'll find that Python not only includes all the interfaces we need in order to write system tools, but also fosters script

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

This chapter begins our in-depth look at ways to apply Python to real programming tasks. In this and the following chapters, you'll see how to use Python to write system tools, GUIs, database applications, Internet scripts, web sites, and more. Along the way, we'll also study larger Python programming concepts in action: code reuse, maintainability, object-oriented programming (OOP), and so on.

In this first part of the book, we begin our Python programming tour by exploring the systems application domain— scripts that deal with files, programs, and the general environment surrounding a program. Although the examples in this domain focus on particular kinds of tasks, the techniques they employ will prove to be useful in later parts of the book as well. In other words, you should begin your journey here, unless you are already a Python systems programming wizard.

Python's system interfaces span application domains, but for the next five chapters, most of our examples fall into the category of system tools—programs sometimes called command-line utilities, shell scripts, and other permutations of such words. Regardless of their title, you are probably already familiar with this sort of script; these scripts accomplish such tasks as processing files in a directory, launching test scripts, and so on. Such programs historically have been written in nonportable and syntactically obscure shell languages such as DOS batch files, csh, and awk.

Even in this relatively simple domain, though, some of Python's better attributes shine brightly. For instance, Python's ease of use and extensive built-in library make it simple (and even fun) to use advanced system tools such as threads, signals, forks, sockets, and their kin; such tools are much less accessible under the obscure syntax of shell languages and the slow development cycles of compiled languages. Python's support for concepts like code clarity and OOP also help us write shell tools that can be read, maintained, and reused. When using Python, there is no need to start every new script from scratch.

Moreover, we'll find that Python not only includes all the interfaces we need in order to write system tools, but also fosters script

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

We will take a quick tour through the standard library sys and os modules in the first few sections of this chapter before moving on to larger system programming concepts. As you can tell from the length of their attribute lists, both of these are large modules (their content may vary slightly per Python version and platform):

>>>import sys, os
>>> len(dir(sys))          # 56 attributes
56
>>> len(dir(os))           # 118 on Windows, more on Unix
118
>>> len(dir(os.path))      # a nested module within os
43

As I'm not going to demonstrate every item in every built-in module, the first thing I want to do is show you how to get more details on your own. Officially, this task also serves as an excuse for introducing a few core system scripting concepts; along the way, we'll code a first script to format documentation.

Most system-level interfaces in Python are shipped in just two modules: sys and os. That's somewhat oversimplified; other standard modules belong to this domain too. Among them are the following:

glob: For filename expansion
socket: For network connections and Inter-Process Communication (IPC)
thread and queue: For concurrent threads
time: For accessing system time details
fcntl: For low-level file control

In addition, some built-in functions are actually system interfaces as well (e.g., open). But sys and os together form the core of Python's system tools arsenal.

In principle at least, sys exports components related to the Python interpreter itself (e.g., the module search path), and os contains variables and functions that map to the operating system on which Python is run. In practice, this distinction may not always seem clear-cut (e.g., the standard input and output streams show up in sys, but they are arguably tied to operating system paradigms). The good news is that you'll soon use the tools in these modules so often that their locations will be permanently stamped on your memory.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

On to module details; as mentioned earlier, the sys and os modules form the core of much of Python's system-related tool set. Let's now take a quick, interactive tour through some of the tools in these two modules before applying them in bigger examples. We'll start with sys, the smaller of the two; remember that to see a full list of all the attributes in sys, you need to pass it to the dir function (or see where we did so earlier in this chapter).

Like most modules, sys includes both informational names and functions that take action. For instance, its attributes give us the name of the underlying operating system on which the platform code is running, the largest possible integer on this machine, and the version number of the Python interpreter running our code:

C:\...\PP3E\System>python
>>> import sys
>>> sys.platform, sys.maxint, sys.version
('win32', 2147483647, '2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)]')
>>>
>>> if sys.platform[:3] == 'win': print 'hello windows'
...
hello windows

If you have code that must act differently on different machines, simply test the sys.platform string as done here; although most of Python is cross-platform, nonportable tools are usually wrapped in if tests like the one here. For instance, we'll see later that today's program launch and low-level console interaction tools vary per platform—simply test sys.platform to pick the right tool for the machine on which your script is running.

The sys module also lets us inspect the module search path both interactively and within a Python program. sys.path is a list of strings representing the true search path in a running Python interpreter. When a module is imported, Python scans this list from left to right, searching for the module's file on each directory named in the list. Because of that, this is the place to look to verify that your search path is really set as intended.

The sys.path list is simply initialized from your PYTHONPATH setting—the content of any .pth path files located in Python's directories on your machine plus system defaults—when the interpreter is first started up. In fact, if you inspect

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

As mentioned, os is the larger of the two core system modules. It contains all of the usual operating-system calls you may have used in your C programs and shell scripts. Its calls deal with directories, processes, shell variables, and the like. Technically, this module provides POSIX tools—a portable standard for operating-system calls—along with platform-independent directory processing tools as the nested module os.path. Operationally, os serves as a largely portable interface to your computer's system calls: scripts written with os and os.path can usually be run unchanged on any platform.

In fact, if you read the os module's source code, you'll notice that it really just imports whatever platform-specific system module you have on your computer (e.g., nt, mac, posix). See the os.py file in the Python source library directory—it simply runs a from* statement to copy all names out of a platform-specific module. By always importing os rather than platform-specific modules, though, your scripts are mostly immune to platform implementation differences. On some platforms, os includes extra tools available just for that platform (e.g., low-level process calls on Unix); by and large, though, it is as cross-platform as it is technically feasible.

Let's take a quick look at the basic interfaces in os. As a preview, Table 3-1 summarizes some of the most commonly used tools in the os module organized by functional area.

Table 3-1: Commonly used os module tools
Tasks	Tools
Shell variables	os.environ
Running programs	`os.system`, `os.popen`, `os.popen2`/`3`/`4`, `os.startfile`

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Python scripts don't run in a vacuum. Depending on platforms and startup procedures, Python programs may have all sorts of enclosing context; information automatically passed in to the program by the operating system when the program starts up. For instance, scripts have access to the following sorts of system-level inputs and interfaces:

Current working directory: os.getcwd gives access to the directory from which a script is started, and many file tools use its value implicitly.
Command-line arguments: sys.argv gives access to words typed on the command line that are used to start the program and that serve as script inputs.
Shell variables: os.environ provides an interface to names assigned in the enclosing shell (or a parent program) and passed in to the script.
Standard streams: sys.stdin, stdout, and stderr export the three input/output streams that are at the heart of command-line shell tools.

Such tools can serve as inputs to scripts, configuration parameters, and so on. In the next few sections, we will explore these context tools—both their Python interfaces and their typical roles.

Additional content appearing in this section has been removed.
Purchase this book now or

Table of Contents

Section 2.3.1.1: A database list

Section 2.4.1.1: Test data script