Errata

[Also posted for 5th Ed] Not an error, but for clarity, expand this sentence:
"""
In fact, if you enjoy working harder than you must, you can also code print operations this way:
"""
To the following, adding only the parenthesized text just before the colon, with links to the two referenced chapters (and please let me know if this changes page breaks; I'm hoping to avoid that):
"""
In fact, if you enjoy working harder than you must, you can also code print operations this way (per Chapters 4 and 9, a 3.X-only return value is omitted here):
"""

[Discussion only follows] A reader wrote by email:
>
> This is on page 357 of the Ebook; page numbering doesn't seem
> to match the print edition. Looking at the last example before the
> "Manual stream redirection" subheading in chapter 11:
>
> >>> import sys
> >>> sys.stdout.write('hello world\n')
> hello world
>
> The system call of course returns a result, and when run
> interactively the result is also printed. As such, when running this
> example under Linux an additional line is added:
>
> >>> import sys
> >>> sys.stdout.write('hello world\n')
> hello world
> 12
>
> where the "hello world" is the text written and the "12" is the result
> of the sys.stdout.write() call. I couldn't find any mention of this on
> your or O'Reilly's errata pages. The omission could be deliberate,
> but some people would find the extra line confusing. I would
> expect the Windows version would add a similar line. There is
> also no mention of this in the following text (i.e. that print always
> returns None where write() returns the number of characters
> written).

Sure, but this isn't platform specific, and is probably not an issue for most readers. The return value of the file's write method is well documented in general file coverage earlier in the book -- starting on Page 122, and spanning Chapters 4 and 9, including the blanket note on Page 288 that reads this way:

"... (for space, I'm omitting byte-count return values from write methods from here on):".

Still, this seems a potential point of confusion, especially for non-linear readers using 3.X. To clarify, I'm suggesting the addition of another parenthetical note about the deliberate omission of the return value in this section, as described above. Such reminders risk becoming overly redundant, but this one seems justified.

Change the sentence at the top of this page:

"""
With OOP, the state remembered is made explicit with attribute assignments.
"""

to the following (with "nonlocal" in literal font):

"""
With OOP, the state remembered is made explicit with attribute assignments; this makes its model generally the most Pythonic, though 3.X's nonlocal statement makes enclosing scopes a viable alternative in some roles.
"""

I'm changing this because the text preceding this suggests that classes are best for state retention, but doesn't clearly explain why, and may be a bit too absolute given the growing popularity of enclosing scope state in Python (formally known as closures, and representative of a trend towards functional techniques).

Reprints: It looks like there is space for a line or two at the end of this page; please ask how to shorten if it changes paging.

I wish to change this sidebar's sentence:

"""
On the other hand, many new learners can pick up advanced topics as needed.
"""

to this:

"""
On the other hand, Python is still much simpler than most of its contemporaries and perhaps only as complex as its many roles require, and many new learners can pick up advanced topics as needed.
""""

This adds two crucial points -- that Python is still a lot simpler than Java, C#, or C++ today despite its growth, and that much of the recently added complexity might be warranted given its many application roles -- and in retrospect the sidebar may read a bit more negatively than intended without these, especially given its end-of-book appearance.

Reprints: It looks like there's room on the page for this, but please contact me if the insert makes the sidebar spill over onto another page (it should be shortened if so, but I'm guessing it takes up just 1 empty line on this page).

I want to clarify that an __init__ in an application class is completely independent of an __init__ in an metaclass; the former runs at class construction time, and the latter at instance construction time, and not by the metclass's __init__. This is all implied by Python's class model (and should probably be intuitive at this point in the book), but to make this key point a bit more explicit, let's change this paragraph from:

"""
In this case, the class initialization method is run after the class construction method, but both run at the end of the class statement before any instances are made:
"""

to the following, adding just the parenthesized text at the end (and making its "Spam" and two "__init__" literal font):

"""
In this case, the class initialization method is run after the class construction method, but both run at the end of the class statement before any instances are made (conversely, an __init__ in Spam would run at instance creation time, and is not affected or run by the metaclass's __init__):
"""

Reprints: It looks like there are 2 free lines on this page, but let me know if this needs trimming to avoid changing page breaks.

[Jul-29-11] More on right-side operator methods: __radd__ = __add__

At the empty space at the bottom of page 728, after the final code listing on this page, add a new short paragraph that reads as follows with all __radd__ and __add__ in literal font (it looks like there's ample room, but please ask how to shorten of not):

"""
For truly commutative operations which do not require special-casing by position, it is also sometimes sufficient to alias the right-side __radd__ to the left-side __add__, by simply assigning the former name to the latter at the top-level of the class statement. Right appearances will then trigger the single, shared __add__ method passing the right operand to self.
"""

For more details on this clarification with examples, see:
http://www.rmi.net/~lutz/lp4e-updates-clarifications-recent.html#radd

The clause "since dir also returns inherited..." seems like a non sequitur. My distillation of "Instance slots" was: In 2.6 __slots__ would be a meaningless instance attribute (that happens to be a list), but in 3.0 it will suddenly take on meaning and preclude arbitrary attribute names, but you can always bypass that preclusion by putting __dict__ in __slots__, and then you should be careful to use getattr to abstract out all the twists.

Note from the Author or Editor:
(This is now the first non-code paragraph on page 794.) The point here is the dir() also includes inherited attrs, and code that wants just the instance's attrs must use a different scheme. This seems fairly clear given the context (note the word "also" for dir()), but to clarify, change the start of this paragraph from:

"Code that wishes to list all instance attributes generically"

to this (add the word "just"):

"Code that wishes to list just all instance attributes generically"

Change:
"and all classes (and hence types) inherit from object."

by adding text at the end to read:
"and all classes (and hence types) inherit from object, which comes with a small set of default operator overloading methods."

This is described ahead on Page 787 and in the context of other examples, but it seems important enough to mention in this summary (and it looks like there is ample space on this page). The default methods of object in new-style classes such as __str__ can sometimes be problematic if not anticipated.

[Nov-22-10] Pages 767, 786: more on new-style inheritance method resolution order (MRO)

First, on page 767, at the end of the very last paragraph before the note box on this page, add the following new sentence ("class.mro()" in both of the text inserts should be literal font): " For more ideas, see also Python manuals for the class.mro() new-style class object method, which returns a list giving the class tree search order used by inheritance; this could be used by a class lister to show attribute sources. ".

Second, at the very end of the last paragrph on page 786, add a new sentence which reads: " To trace how new-stye inheritance works by default, see also the class.mro() method mentioned in the preceding chapter's class lister examples. ".

----
[For a full discussion of this topic, please see my main errata page from which this change was copied:
http://www.rmi.net/~lutz/lp4e-updates-clarifications-recent.html]

Sep-29-10] Page 27 and 534, note "magic" version number check for byte-code recompiles

1) On page 27, at the very end of the 2nd paragraph that begins "Python saves", add this sentence: "Imports also check to see if the file must be recompiled because it was created by a different Python version, using a "magic" number in the byte-code file itself."

2) Also, on page 534, at the very end of the second to last paragraph which begins "Python checks", add another sentence: "As noted in Chapter 2, imports also recreate byte code if its "magic" Python version number does not match."

Discussion: Technically, in order to know if a recompile is required, imports check both source/bytecode timestamps as well as the bytecode file's internal version/implementation "magic" number The book describes the timestamp check because it's fundamental to all users, but does not go into further detail about the extra magic number test because this is arguably more low-level and detailed than most Pyton newcomers using a single Python require. It becomes more important to know once you start installing new or alternative Pythons, of course, though it would be difficult to imagine how an incompatible Python could work at all without such a mechanism.

[from http://www.rmi.net/~lutz/lp4e-updates-clarifications-recent.html]

[Sep-27-10] Chapter 23: The 3.X package-relative import model precludes using directories as both program and package

I posted a clarification about issues related to inter-package imports under the Python 3.X package-relative import model on my main updates page. Since it's too long to paste here, please see the original page for this item, and its suggestion book inserts:

http://www.rmi.net/~lutz/lp4e-updates-clarifications-recent.html

(Crossposted from http://www.rmi.net/~lutz/lp4e-updates.html for reprint inclusion)

[Sep-1-10] Page 976 and 956: Note that descriptor state cannot vary per client class instance

Add a new sentence at the very end of paragraph 3 on pge 976, which reads "The downside of this scheme is that state stored inside a descriptor itself is class-level data which is effectively shared by all client class instances, and so cannot vary between them.".

Also, after the first sentence of the last paragraph on page 956, add a new sentence which reads "Unlike data stored in the descriptor itself, this allows for data that can vary per client class instance.". It looks like there is space for both inserts, but please ask me how to shorten if not.

Rationale: There is an implication of descriptor state options which might have been called out more explictly than it was. Crucially, storing state in the descriptor instance instead of the owner (client) class instance means that the state will be effectively shared by all owner class instances. That is, because descriptors are class-level data, their content cannot vary per instance of client classes.

To see this at work, in the descriptor-based CardHolder example on page 976-977, try printing attributes of the "bob" instance after creating the second instance, "sue". The values of sue's managed attributes ("name", "age", and "acct") effectively overwrite those of the earlier object bob, because both share the same, single descriptor instance attached to their class:

class CardHolder: ...as is...

bob = CardHolder('1234-5678', 'Bob Smith', 40, '123 main st')
print(bob.name, bob.acct, bob.age, bob.addr)

sue = CardHolder('5678-12-34', 'Sue Jones', 35, '124 main st')
print(sue.name, sue.acct, sue.age, sue.addr)
# addr differs: cardholder instance data
print(bob.name, bob.acct, bob.age, bob.addr)
# name,acct,age same: descriptor data!

...> C:\Python31\python test.py
bob_smith 12345*** 40 123 main st
sue_jones 56781*** 35 124 main st
sue_jones 56781*** 35 123 main st

There are valid uses for descriptor state, of course (to manage descriptor implementation, for example), and this code was implemented to illustrate the technique. Moreover, the state scope implications of class versus instance attributes should be more or less a given at this point in the book. However, in this particular use case, attributes of CardHolder objects are probably better stored as per-instance data instead of descriptor instance data, perhaps using the same __X naming convention as the property-based equivalent to avoid name clashes in the instance.

(A reader wrote to ask how code snippets on page 333 could
be made to actually run; my reply/clarification is below.
If we have space to patch in reprints, add the text "(this
code is intentionally abstract and incomplete: x is a sequence
and match() is a tester function to be defined)" just before
the first code listing on this page)

Thanks for your note. You have a valid point: this code snippet
is intended to be abstract and partial, but it's not explicitly
described as such. In fact, the prior page's primes code is
similarly abstract, though this fact is better noted there.

To make it work as real, live code, "x" would have to be a sequence
such as a list or string, and "match()" would have to be a function
which checks for a match against an object passed in. As a
completely artificial example:

x = list(range(100))
def match(I):
return I > 50

A better example might make x a list of dictionary "records",
and match() a test against a dictionary key's "field" value
(e.g., looking for a record with a name value known to match()).

The abstract code snippet in the book strips off successive items
at the front of "x" and passes each into this function in turn.
I couldn't show a function like match() at this point in the
book, though, without yet another forward dependency (functions
are not covered until the next part). The goal here was to illustrate the loop else by itself.

I also chose not to elaborate here because in practice a "for" loop
is probably better than a "while" for this code, and iteration
tools such as filter() and list comps might be better than both:

x = [....]
for item in x:
if match(item):
print('Ni')
break
else:
print('Not found')

x = [....]
print('Ni' if [item for item in x if match(item)] else 'Not found')

x = [....]
print('Ni' if filter(match, x) else 'Not found')

Try running these alternatives on your own to see what I mean.

This is for 4th edition.

I have a question/request for clarification for self.name.upper() in the context of the text below:

Even methods, normally created by a def nested in a class, can be created completely independently of any class object. The following, for example, defines a simple function outside of any class that takes one argument:

>>> def upperName(self):
... return self.name.upper() # Still needs a self

There is nothing about a class here yet?it?s a simple function, and it can be called as such at this point, provided we pass in an object with a name attribute (the name self does not make this special in any way):

====

My question:

I am lost about self.name.upper(). Why is this self.name.upper() instead of simply self.upper()?

From the context, 'name' is an attribute of object x and also an attribute of class rec. How can this 'name' attribute have an attribute (the upper() function) of its own? Is it a "nested attribute"? Is there even such a thing in Python?

Maybe I completely misunderstood. Could you please clarify this particular line of code?

Thank you!

Note from the Author or Editor:
Well, the code is correct as shown, but the "self" in it might be a bit confusing (it's just a simple variable name here).

I'd call this nested objects, not nested attributes. To understand it fully, you must evaluate it the way Python does--from left to right, and one expression/operation at a time. Given "self.name.upper()", and adding parenthesis to emphasize the order of operations:

1) (self.name) fetches the value of a "name" attribute from whatever object variable "self" happens to reference.
2) ((self.name).upper) then fetches the value of an "upper" attribute from whatever object was returned by step 1.
3) ((self.name).upper)() finally calls the function object that "upper" is assumed to reference, with no arguments.

The net effect is that "self" references an object, whose "name" attribute references a (string) object, whose "upper" attribute references a (callable) object. It's object nesting; in general, that's what class instance state information always is--nested objects assigned to instance attibutes.

And that why it works to pass "x" to this function directly: "x" is a class instance object, whose "name" attribute references a string object with an "upper"; "x" has no "upper" attribute itself.

Thanks for your note. We don't have space to add text on this page, but hopefully this clarification will help other readers too.

re:
To understand this code, it's crucial to notice that the attribute assignments inside the
__init__ constructor method trigger property setter methods too.

python 2.6.5, linux

Stepping with pydev debugger through Attribute_validation_w_properties it appears
instance attribute assignments are only intercepted by properties for re-assignments, eg. bob.name = 'Bob Q. Smith' but not during instatiation since self._name remains 'Bob Smith" not "bob_smith" as setter implies. Correct ???

inside __init__ "name mangling" missing out and perhaps just one leading underscore ?

self._acct
self._name
self._age
self.addr

Excellent chapter though. For time I eventually managed to grasp the mysteries behind getter, setter etc and how they differ and start to get an idea what situation ea fit best.
Many Thanks
Mario Dix

Note from the Author or Editor:
No, the example does work as shown and described, and the "__name" attributes format is intended. However, this is arguably one of the most subtle examples in the book, so I'll try to clarify a bit here.

To see that the setter is indeed called for assignments in __init__ at instance creation time, try adding a print() in the setter methods,
and either run the self-test code or import the class and create an instance interactively:

class CardHolder...
def setName(self, value):
print('in setName')

>>> CardHolder('11111111', '25', 3, '44')
in setName
<test.CardHolder object at 0x01410830>

The setter is called from __init__ when the instance is first created and the attribute is assigned, under both Python 3.X and 2.X. Also make sure that you derive the class from "object" under 2.X to make it a new-style class. As explained earlier in this chapter (and in Chpater 31), property setters don't quite work under 2.X without including "object" in the superclass list; once an attribute name is mistakenly assigned directly on an instance, it hides the property getter in the class too (perhaps this was the entire issue here?):

class CardHolder(object): # required in 2.X

With this change results under 2.6 and 3.1 are identical. You'll also need to use 2.X-style print statements or a from __future__ for 3.X-style print calls; see earlier in the book for print() in 2.X:

from __future__ import print_function

The other oddness in this example (which is covered earlier in the book but perhaps not explained as explicitly for this example itself as it could have been) is that names beginning with 2 underscores like "__name" are pseudo-private attributes: Python expands them to include the enclosing class's name, in order to localize them to the creating class. They are used intentionally here to avoid clashing with the real attribute names such as "name" that are part of the class's external client API. Python mangles each in-class appearance of the attribute like this:

__name__ ...becomes... _CardHolder__name

The single underscore naming pattern "_name" used elsewhere in this chapter is a weaker convention that informally attempts to avoid name collisions, but "__name" truly forces the issue, and is especially useful for classes like this one which manage attribute access but also need to record real state information in the instance. Clients use "name" (the property), and the expanded version of "__name" (the data) where state is actually stored is more or less hidden from them. Moreover, unlike "_name", it won't clash with other normal instance attributes if this class is later extended by a subclass.

Thanks for your note and feedback; I'll mark this as "confirmed" so the clarification is available to others on O'Reilly's errata page.

Reprints: a reader posted a nice list of Index additions which I added clarifications to and confirmed, but this post does not seem to be showing up here as far as I can tell. I saved the original--please ask me for it if you can't find this when updating for a reprint. --Mark Lutz

In order to be useful as a reference manual (I read the entire book, now I need to refer to it) several items need to be added to the index. When I don't find something in the index when I am looking, when I eventually find it I write it in the index. Each of these are of this nature.

For each item, I give the text and the page numbers. Some index text is already present, the page numbers are added to the corresponding index entry.

* 133, 185, 284
** 189
% (formatting operator), 187
__all__ variable, 564
all, any, 364, 503
bool type, 322
classtools.py 665
dbm module, 216
diamond pattern of multiple inheritance trees, 778
eggs, 889
Emacs, 64
enumerate function, 358
environment variables, 1093
eval function, see also exec
Exception class, 830, 838, 858
exec function, see also eval
getattr, 661
import statement, as keyword, 566, 591
isinstance 468, 724
lock (thread), 853
min and max functions, 364
nop, see pass
operator precedence, 109
Perl 183
precedence, 109
private, 584, 720
regex, 85
singleton, 985
static methods, 753
strip, see rstrip
sys.exec_info, 854, 860
syst.exit, 883
threading, 853
traceback, 881
vars 183
with statement, 238
zip, 218

Note from the Author or Editor:
Thanks for your list. Although this book explicit describes itself as tutorial instead of reference, the Index could use some tweaking. It's gotten better in recent editions, but this has histrically been difficult to do well in the absence of reader feedback such as yours (O'Reilly does indexes itself as part of its production process).

I recommend adding as many of the items listed in this report as space allows, apart from the following exceptions and clarifications (which I hope format readably in O'Reilly's errata page):

--the "*" noted is for argument upacking, not the repitition operator (which is aready indexed).

--the '**' noted is for argument unpacking, not power (not indexed).

--"getattr" also should list page 592.

--"import statement, as keyword" should give the "as" in italics or quotes (that's the main point of this entry).

--"isinstance" should also include pages 101 and 250.

--"% (formatting operator)" should also include page 179.

--"regex" is already indexed with this page number under "re", the name of Python's regex module; instead, add a "regex, see re" entry.

--"sys.exit" is already indexed (though along with its argument too, unusually); your "syst.exit" looks like a typo.

--"Perl" doesn't appear on the page listed; it does on pages 18 and 20.

--"private" should also include page 1023, where a full-blown privacy implementation is hashed out.

The real limiting factor on these is the limited space we have for inserts in reprints; they can't shift page content much if at all. For what it's worth, I also wrote a book "Python Pocket Reference 4E"
to provide the quick lookup resource you may find useful after learning the basics.

(Clarification added for inclusion in reprints from
http://www.rmi.net/~lutz/lp4e-updates-clarifications-recent.html)
------

[Jul-7-10] Page 936, last sentence of page: Unicode -- mention filename tools too (new text)

Change the last part of the text: "For more details on re, struct, pickle, and XML tools in general, consult" to read: "For more details on re, struct, pickle, and XML, as well as the impacts of Unicode on other library tools such as filename expansion and directory walkers, consult".

The section here dealing with tools impacted by Unicode could also have mentioned that os.listdir returns decoded Unicode str for str arguments, and encoded raw binary bytes for bytes arguments, in order to handle undecodable filenames. In short, pass in the directory name as a bytes object to suppress Unicode decoding of filenames per the platform default, or else an exception is raised if any filenames fail to decode. Passing in a str invokes Unicode filename decoding on platforms where this matters.

By proxy, os.walk and glob.glob work the same way, because they use os.listdir internally to generate filenames in directories. This was omitted here because the section already encroaches on the language/applications line. Instead, the impacts of Unicode on these and other tools are covered in depth in the new 4th Edition of Programming Python, where application topics are collected in general.

(Clarification added for inclusion in reprints from
http://www.rmi.net/~lutz/lp4e-updates-clarifications-recent.html)
------

[Jul-2-10] Page 909: Unicode -- "conversion" means encoding differently (new sentence)

At the very end of the last paragraph on this page, add the following new sentence: "Either way, note that "conversion" here really just means encoding a text string to raw bytes per a different encoding scheme; decoded text has no encoding type, and is simply a string of Unicode code points (a.k.a. characters) in memory.".

(Clarification added for inclusion in reprints from
http://www.rmi.net/~lutz/lp4e-updates-clarifications-recent.html)
------

[Jul-7-10] Page 901, start of 1st paragraph on page: Unicode -- same policy for read, write (reword)

The start of this paragraph seems potentially misleading in retrospect--it's not clear if writes work the same as reads. This is clarified later on (see page 920 and later), but it may be worth tightening up here.

Change: "When a file is opened in text mode, reading its data automatically decodes its content (per a platform default or a provided encoding name) and returns it as a str; writing takes a str and automatically encodes it before transferring it to the file."

to read as this (the parenthesized part has been pulled out): "When a file is opened in text mode, reading its data automatically decodes its content and returns it as a str; writing takes a str and automatically encodes it before transferring it to the file. Both reads and writes translate per a platform default or a provided encoding name."

(Clarification added for inclusion in reprints from
http://www.rmi.net/~lutz/lp4e-updates-clarifications-recent.html)
------

[Jul-2-10] Page 900: Unicode -- internal str format (new footnote)

I avoided internals discussion in this chapter on purpose, using terms such as "character" instead, but in retrospect some readers might find a more tangible model useful too. Add a foonote at the bottom of page 900, with its star at the very end of the last paragraph before header "Text and Binary Files", which reads:

"It may help to know that Python internally stores decoded strings in UTF-16 (roughly, UCS-2) format, with 2 bytes per character (a.k.a. Unicode "code pont"), unless compiled for 4 bytes/character. Encoded text is always translated to and from this internal form, in which text processing occurs.".

Reprints: if this doesn't fit at the bottom of this page as is, please ask me how it could be shortened

(Clarification added for inclusion in reprints from
http://www.rmi.net/~lutz/lp4e-updates-clarifications-recent.html)
------

[Jul-2-10] Page 899: Unicode -- bytes is for encoded str too (new text)

At the second bullet item in the second bullet list on this page, add the following text in parenthesis at the end, so that the bullet item reads: "* bytes for representing binary data (including encoded text)". This is shown and implied in later examples, but this seems like a key link concept.

(Clarification added for inclusion in reprints from
http://www.rmi.net/~lutz/lp4e-updates-clarifications-recent.html)
------

[Jul-2-10] Page 898: Unicode -- mention UTF-16 and UTF-32 in intro (new text)

Near the end of the second last paragraph on this page, expand the start of the second last line by adding the parenthesized text in the following, to read: "sets in similar ways (e.g., UTF-16 and UTF-32 format strings with 2 and 4 bytes per each character, respectively), but all of these". This is implied by later UTF-16 examples, but UTF-16 is so common on Windows now that it merits a word here.

(Clarification added for inclusion in reprints from
http://www.rmi.net/~lutz/lp4e-updates-clarifications-recent.html)
------

[Jul-7-10] Page 896, end of second last paragraph: Unicode -- clarify impacts (new sentence)

At the very end of the paragraph which begins "Even if you fall into", add a new last sentence which reads: "Though applications are beyond our scope here, especially if you work with the Internet, files, directories, network interfaces, databases, pipes, and even GUIs, Unicode may no longer be an optional topic for you in Python 3.X."

I'm adding this because the existing text seems a bit misleading, after seeing firsthand how much Unicode permeates 3.X applications work. See this note for related discussion. Reprints: delete the first clause of this new sentence of it won't fit as is; it looks like there is plenty of room.

(This and 6 other Unicode items on this page arose from a recent reread of the Unicode chapter a year after writing it; it's fine as is, but a few key concepts could be polished with simple inserts in the next printing.)

The output tuple ('0100', '0x40, '0b1000000'). Shouldn't the first Octal literal be be '0o100'?

The printed value in the book is correct for Python 2.6, but not 3.1.1

Note from the Author or Editor:
Yes, you're right--this should be changed to 0o100.

The new 3.X format for octals is described in depth in this section, but this specific example was run under 2.6; since the rest of the book is "3.X unless stated otherwise", this should be patched in reprints. Oddly, another 2.6 item snuck in on the next page (an already-posted errata); these were probably run at the same time.

[Dec-15-09] Page 511, note decorator-based timer alternatives in ch38 and ch39 (add new footnote)

In Chapter 20, the book explores general timing functions whih require special call patterns for the functions timed. Later in the book, decorator-based timer alternatives are developed which don't require special call syntax, and it would be nice to have a forward link to the alternative coding here.

Add the following quoted text as new footnote on page 511, with its asterisk at the very end of the paragraph just before heading "Timing Results" [reprints: it looks like we have space, but please ask how to shorten if not]:
"""
Also note how we must pass functions in to the timer manually here. In Chapters 38 and 39 we'll see decorator-based timer alternatives with which timed functions are called normally.
"""

--------
crossposted from http://www.rmi.net/~lutz/lp4e-updates.html

[Dec-14-09] Page 513-518, iteration timings, map() is slow because all function calls are slow (add new footnote)

Add the following text as a new footnote on page 518 (reprints: it looks like we have space on this page but please ask how to shorten if not). Its asterisk should appear at the very end of the last paragraph before heading "Function Gotchas" on page 518, and in this the words "map" and "abs" and the code "def f(I): return(I)" should all be fixed-with font:

""" For more fun, apply a simple user-defined function like def f(I): return(I) in all five iteration techniques timed. The results are similar to using a built-in function like abs: among the five iteration techniques, map is fastest today if all five call a function, built-in or not, but slowest when the others do not. That is, map appears to be slower simply because it requires function calls, and function calls are relatively slow in general. Since map can't avoid calling functions, it can lose by association. """


[Discussion follows -- no further changes required]

This section goes through a code timing example and gives the relative speeds of various iteration alternatives. The map built-in appears to win when a built-in function is applied, and lose to list comprehensions otherwise. All true, but this section wasn't meant as a complete treatise on Python performance, and some additional conclusions were not explored in the interest of space. Among these is a more general further conclusion about map speed which I'd like to note here, and briefly footnote in the book.

Technically, map is relatively slower because it requires function calls, and function calls are expensive in general. It doesn't matter if the function called is built-in or user-defined. To see this for yourself, try changing the timing examples to apply a simple user-defined function; the results are the same as using a built-in function like abs -- map is faster if all five techniques call functions of any variety. Unlike list comprehensions and for loops, though, map can't avoid the cost of function calls, so it is slowed by association. At least that's the case today. As also noted in the book, timing results can vary per release and other variables, so be sure to time code yourself to be sure. Performance measurement is often part black art.

[Dec-8-09] Page 1119, generator-based alternative to file counts exercise solution

If we have space, add the following sentence to the start of the last paragraph on page 1119: "A generator expression can have much the same effect: sum(len(line) for line in open(name)).", where the second half of this, "sum(...)", is code in fixed-width font.

[discussion follows -- no further reprint changes required]

I'd like to add this because this alternative generator solution is as nice as the one shown, though arguably less explicit:

# shown: loads into memory all at once, fails for large files

>>> len(open('README.txt').read())
7013

# shown: reads one line at a time with file iterators

>>> tot = 0
>>> for line in open('README.txt'): tot += len(line)

>>> tot
7013

# not shown: generator expression -- reads one line at a time, sums along the way

>>> sum(len(line) for line in open('README.txt'))
7013

# other part: using binary mode to match system size (keep Windows \r\n)

>>> sum(len(line) for line in open('README.txt', 'rb'))
7216
>>> import os
>>> os.path.getsize('README.txt')
7216

See Chapter 20 for more on generator expressions, and Chapters 4, 5, and 14 for sum(). sum() allows any iterable, and both the file iterator and generator expression in this alternative solution defer file line fetches until requested. The effect is to avoid an explicit for loop, and perhaps speed performance.

--------
This clarification was crossposted from the following web site:
http://www.rmi.net/~lutz/lp4e-updates.html

This an errata about O'Reilly's errata system (a meta-errata?).

O'Reilly's errata tracking system (this site) strips whitespace in program code that is posted to it. Unfortunately, because whitespace is signicant to the left of code in Python, and a required part of Python syntax, some of the code posted here is not correct.

For instance, the "def" and "class" statements in posts here are invalid as shown, and you'll have to infer the correct indentation from their content. Less harmful but still painful, space has been stripped before "#" comments, leading to very unreadable code. Not exactly a Python-friendly interface.

I've asked O'Reilly to look into this; for now, when in doubt please refer to the correctly presented code on my website:

http://www.rmi.net/~lutz/lp4e-updates.html

(This may also be an issue for PDF files generated by O'Reilly for Python titles: it appears that whitespace does not survive cut-and-paste of code. This is TBD, pending a review by O'Reilly.)

--Mark Lutz

[Nov-07-09] Page 1141, Index, need entry for bitwise operations

The index really should include more about bitwise processing of integers; as is, there are links to page 108 for operator symbols only, but this is incomplete.

Add an entry to the index, "bitwise operators (binary data)", with links to pages "108, 124-125, 930-932". If this won't fit on the page, shorten the title to "bitwise (binary) data" and/or make the links "108, 124, 932".

--------
This item is part of a batch that was cut-and-paste from the
following webpage, which may contain additional information
about the issue:

http://www.rmi.net/~lutz/lp4e-updates.html

[Nov-12-09] Page 910, printing 2.6 unicode, display of non-ASCII str can vary

At the very end of the first paragraph on this page, change: "(all other sections in this chapter are run under 3.0):" to "; unicode characters display in hex in 2.6 unless you explicitly print, and non-ASCII displays can vary per shell (most of this section ran in IDLE):". [Reprints: please ask me to shorten if this doesn't fit as is.]

[Discussion only follows -- no further reprint changes required]

I'm changing this because: (1) The original isn't quite correct -- there are other 2.6 sessions in this chapter; (2) The examples imply that echoes (repr) display hex for 2.6 Unicode, but "print" displays characters -- this isn't stated explicitly, though; and (3) The display of the first test in this section varies between IDLE and a DOS shell interface on Windows.

I ran this in IDLE and got the results shown in the text, but a non-IDLE user might be confused. Moreover, a later example gives the DOS display format, and this all differs in 3.X, where all strings are Unicode and always display characters:

# Python 2.6, in the IDLE GUI (same as shown in book)

>>> S = 'A\xC4B\xE8C'
>>> S
'A\xc4B\xe8C'
>>> print S
A?B?C
>>>
>>> U = u'A\xC4B\xE8C'
>>> U
u'A\xc4B\xe8C'
>>> print U <=== print in 2.X to show chars
A?B?C

# Python 2.6, in a Windows Command Prompt (DOS shell)

>>> S = 'A\xC4B\xE8C'
>>> S
'A\xc4B\xe8C'
>>> print S
A&#9472;B&#934;C <=== DIFFERS from IDLE display
>>>
>>> U = u'A\xC4B\xE8C'
>>> U
u'A\xc4B\xe8C'
>>> print U
A?B?C

# Python 3.1, in both interfaces (IDLE, DOS shell)

>>> S = 'A\xC4B\xE8C'
>>> S <=== no need to print() in 3.X
'A?B?C'
>>> print(S)
A?B?C
>>>
>>> B = b'A\xC4B\xE8C'
>>> B
b'A\xc4B\xe8C'
>>> print(B)
b'A\xc4B\xe8C'

--------
This item is part of a batch that was cut-and-paste from the
following webpage, which may contain additional information
about the issue:

http://www.rmi.net/~lutz/lp4e-updates.html

[Oct-28-09] Page 777, 2.6 classes must derive from object _explicitly_

In the second bullet on this page, I'd like to underscore that the derivation from "object" in Python 2.6 new-style classes must be explicit. Change "In Python 2.6 and earlier, classes must inherit from object" to "In Python 2.6 and earlier, classes must explicitly inherit from object".

This is implied by the following code example, but the "explicit" should be made (well...) explicit, since the preceding bullet does so.

--------
This item is part of a batch that was cut-and-paste from the
following webpage, which may contain additional information
about the issue:

http://www.rmi.net/~lutz/lp4e-updates.html

[Nov-11-09] Page 728, Bound methods already defer calls -- avoid superfluous lambdas!

Insert a sentence at the start of the 5th paragraph on page 728, so that it reads: "Note that a lambda is not required here, because a bound method reference by itself already defers a call till later (again, more on bound methods in Chapter 30). This technique is simpler, ...". It look's like there is plenty of space on this page.

[Discussion only follows -- no further reprint changes required]

I'm adding this to underscore a point about deferred code which is already covered by the book but is an apparently common stumbling block. This section, as well as the more complete coverage in Chapter 30 with its sidebar on page 756, show how bound methods, named through self, may be used as callback handlers, because they are callable objects.

For instance, the following registers self.handler as the event handler for a tkinter GUI event. As explained in the book, referencing the method without calling it like this (i.e., without coding parenthesis after it) is sufficient to defer its call until the event later occurs and invokes a call:

>>> from tkinter import *
>>>
>>> class MyGui:
... def __init__(self):
... self.color = 'Blue'
... Button(text='Spam', command=self.handler).pack(side=TOP)
... def handler(self):
... print('Got it:', self.color)
...
>>> X = MyGui()
>>> mainloop() # and press button in GUI
Got it: Blue
Got it: Blue

Recently, though, I've seen the following sort of variation in a variety of Python code on the web -- using a lambda to defer a method call. This shows up even in tutorials for popular Python frameworks. But the lambda in this is completely pointless -- if you simply leave the parenthesis off, self.handler calls are already deferred:

>>> class MyGui:
... def __init__(self):
... self.color = 'Blue'
... Button(text='Spam',
... command=(lambda: self.handler())).pack(side=TOP)
... def handler(self):
... print('Got it:', self.color)
...
>>> X = MyGui()
>>> mainloop()
Got it: Blue
Got it: Blue

Both versions work, but the lambda in the second is superfluous overkill, and probably reflects a lack of knowledge about Python's bound methods (if not bad habits from other programming languages). Lambda is required in other contexts that need more than a deferred method call (e.g., when you must also specify arguments to be passed to the method); here, though, it's just extra code, and incurs the performance penalty of an extra call.

This is exactly the sort of subtle mistake that abbreviated Python introductions often miss, and which in-depth books like Learning Python are designed to address and prevent. See, for instance, pages 727-728, 750-756, and 479. The 3rd Edition of the application-focused book Programming Python naturally covers GUI callback coding options as well (especially on pages 390-399).

--------
This item is part of a batch that was cut-and-paste from the
following webpage, which may contain additional information
about the issue:

http://www.rmi.net/~lutz/lp4e-updates.html

Nov-11-09] Page 746, use list() around dict.keys() in 3.X (a missed occurrence)

In the third-last line of the last code listing on this page, change ">>> x.keys()" to ">>> list(x.keys())".

Without the list(), the output of this line shown is produced in Python 2.X, not 3.X. I'm not marking this as an error since the need to wrap key lists in list() for 3.X only is documented in numerous places before this; using list() is superfluous for 2.X readers; and this book covers both 3.X and 2.X. Moreover, the rehash of this example later on page 1013 does use the list() call. Still, list() is used for key lists everywhere else to emphasize 3.X usage, so this occurrence might confuse if not changed.

--------
This item is part of a batch that was cut-and-paste from the
following webpage, which may contain additional information
about the issue:

http://www.rmi.net/~lutz/lp4e-updates.html

[Nov-11-09] Page 670, shelve changes and the "writeback" option

[No reprint fix required -- clarification only]

I want to make a minor clarification about shelves. Page 670 states that you must close shelves after making changes, and page 676 states that ZODB writes changes to disk automatically, implying that shelve does not (also implied by a code comment in the update script on page 674). This is all true by default, and no fix is required here -- this book doesn't cover such libraries in any sort of depth.

However, if the shelve.open() call's optional new writeback keyword argument is passed True, all entries accessed are cached in memory, and written back at close time. This makes it easier to change mutable entries in the shelve, but can consume memory for the cache, and can make the close operation slow because all accessed entries are written back. You still must close the file after changes as stated; the only difference is that with this option you need not manually reassign changed objects to their keys in the shelve in order to write changes to disk.

See Python's library manual, reference books, or application-focused books for more details on shelve.

--------
This item is part of a batch that was cut-and-paste from the
following webpage, which may contain additional information
about the issue:

http://www.rmi.net/~lutz/lp4e-updates.html

[Nov-06-09] Page 431, alternative scope-based state retention technique (add new footnote)

To clarify this in future printings, make two changes:

1) Add a footnote star at the very end of the first paragraph on page 432, referencing a new footnote on page 432 that has the following text: """Function attributes are supported in both Python 2.6 and 3.X. We'll explore them further in Chapter 19, and revisit all the state options introduced here in Chapter 38 in a more realistic context. Also note that it's possible to change a mutable object in the enclosing scope in 2.X and 3.X without declaring its name nonlocal (e.g, state=[start] in tester, state[0]+=1 in nested), though it's perhaps more obscure than either function attributes or 3.X's nonlocal.""". Within this new footnote text, "state=[start]", "tester", "state[0]+=1", and "nested" should all be in fixed-width font.

2) Add page 471 to the index entry for "function attributes" (though it's already mentioned in function/attributes and annotations).

[Reprints: please let me know if these two changes need clarification or trimming.]

[Discussion follows -- no further reprint changes required]

I want to point out an alternative to a book scopes example. The last section on page 431 gives an example of using function attributes to retain per-call state information, as an alternative to the 3.X-only "nonlocal" statement. The following example from the book (which runs in both 2.X and 3.X) works because each call to the outer function runs a nested def to make a new function object; it also allows the state information to be accessed from outside the function if needed:

# Python 2.X and 3.X: function attributes (in book)

>>> def tester(start):
... def nested(label):
... print(label, nested.state)
... nested.state += 1 # changes object, not name
... nested.state = start # per-call state is retained:
... return nested # each tester() makes new func
...
>>> F = tester(0)
>>> F('spam')
spam 0
>>> F('eggs')
eggs 1
>>>
>>> G = tester(42) # each nested func object has its own state
>>> G('ham')
ham 42
>>> G('bacon')
bacon 43
>>>
>>> F('sausage')
sausage 2
>>> F.state, G.state # and state is accessible from outside
(3, 44)

Interestingly, you can achieve the same effect with a reference to a mutable object in the enclosing scope instead of an attribute attached to the function itself, albeit at some arguable cost in code simplicity. The following alternative, not listed in the book, works similarly because each call to the outer function gets a new local scope; unlike function attributes, though, the state is not directly visible outside the function:

# Python 2.X and 3.X: change mutable in enclosing scope (not in book)

>>> def tester(start):
... def nested(label):
... print(label, state[0])
... state[0] += 1 # changes object, not name
... state = [start]
... return nested
...
>>> F = tester(0) # per-call state retained again:
>>> F('spam') # each tester() has new local scope
('spam', 0)
>>> F('eggs') # but no F.state accessible here
('eggs', 1)
>>>
>>> G = tester(42)
>>> G('ham')
('ham', 42)
>>> G('bacon')
('bacon', 43)
>>>
>>> F('sausage')
('sausage', 2)

Of course, as shown in the book, the 3.X "nonlocal" statement offers an alternative that is often seen as more straightforward than both the above schemes; its only potential downside is that unlike function attributes, the state it retains is not directly visible outside the function (which may matter in some contexts):

# Python 3.X only: nonlocal statement (in book)

>>> def tester(start):
... state = start
... def nested(label):
... nonlocal state # change enclosing scope name
... print(label, state) # but state not visible outside
... state += 1
... return nested
...
>>> F = tester(0)
>>> F('spam')
spam 0
>>> G = tester(42)
>>> G('eggs')
eggs 42
>>> F('eggs')
eggs 1
>>> F.state
AttributeError: 'function' object has no attribute 'state'

Also see the further coverage of function attributes in page 471; this probably should be noted at the earlier page 431 example, along with the fact that both 2.6 and 3.X support function attributes, and the rehash of these techniques in the decorators chapter. To clarify all this a bit in future printings, make the two changes described above.

--------
This item is part of a batch that was cut-and-paste from the
following webpage, which may contain additional information
about the issue:

http://www.rmi.net/~lutz/lp4e-updates.html

[Nov-01-09] Page 413, Chapter 17, scopes and loop variables in comprehensions (add new footnote)

To clarify, in future printings of the book we'll add the following text as a footnote at the bottom of page 413, with its star at the very end of the sentence just before the note box on that page [reprints: please ask me how to shorten this if it does not fit on that page as is]: """There is technically one more scope in Python: loop variables in comprehension and generator expressions are local to the expression itself in 3.X (in 2.X, they are local in generators but not in list comprehensions). This is a special and obscure case that rarely impacts real code, and differs from for-loop statements which never localize their variables.""".

[Discussion follows -- no further reprint changes required]

I wish to clarify an arguably obscure scope issue not covered in the book -- that of loop variables in comprehensions. In 2.X, such loop variables are local to generator expression, but not list comprehensions. In 3.X, they are always local to both kids of expressions, as well as 3.X's new set and dictionary comprehensions. This differs from variables in loop statements which are never local to the statement itself. It also seems almost too academic and arcane to mention (and frankly, it's difficult to imagine a valid use case that would depend upon either behavior), but this is technically another 3.X change that could break 2.X code.

Here's how this pans out in practice -- in 3.X comprehensions localize loop variables, but statements don't:

# Python 3.0

>>> L = [x for x in 'spam']
>>> L
['s', 'p', 'a', 'm']
>>> x
NameError: name 'x' is not defined
>>>
>>> G = (y for y in 'spam')
>>> G
<generator object at 0x02501D00>
>>> y
NameError: name 'y' is not defined
>>>
>>> {x for x in 'spam'}
{'a', 'p', 's', 'm'}
>>> x
NameError: name 'x' is not defined
>>>
>>> {x: x*2 for x in 'spam'}
{'a': 'aa', 'p': 'pp', 's': 'ss', 'm': 'mm'}
>>> x
NameError: name 'x' is not defined
>>>
>>> for z in 'spam': print(z, end=' ')
...
s p a m >>> z
'm'

In Python 2.6, list comprehensions and generator expressions are supported, but the former does not localize its loop variables -- they retain their last iteration value, just like for-loop statements:

# Python 2.6

>>> L = [x for x in 'spam']
>>> x <=== DIFFERS
'm'
>>>
>>> G = (y for y in 'spam')
>>> y
NameError: name 'y' is not defined
>>>
>>> for z in 'spam': print z,
...
s p a m
>>> z
'm'

--------
This item is part of a batch that was cut-and-paste from the
following webpage, which may contain additional information
about the issue:

http://www.rmi.net/~lutz/lp4e-updates.html

[Nov-10-09] Page 195, bytearray is mutable too (type summary is informal)

I resisted including both 3.X's bytes and bytearay types and 2.6's unicode type in the datatype lists in this section on purpose, to avoid getting too fine grained or version specific at this point in the book. bytes, bytearray, and 2.6 unicode are kinds of strings; are introduced earlier in this chapter but mostly deferred till Chapter 36; and are included in the later type summaries and figures in Chapter 9 (see pages 240 and 249).

Since bytearray is available in both 2.6 and 3.X, though, we'll clarify this a bit here: at the very first line on page 195, change "Mutables (lists, dictionaries, sets)" to "Mutables (lists, dictionaries, sets, bytearray)".

--------
This item is part of a batch that was cut-and-paste from the
following webpage, which may contain additional information
about the issue:

http://www.rmi.net/~lutz/lp4e-updates.html

[Nov-09-09] Page 186, dict() forward reference (to refer back to page 91 insert)

This is related to the clarification for page 91 earlier on this page. On page 186, at the very end of the 3rd-last paragraph that ends with "10-character-wide field", add this text just before the colon there: "(note the use of dict() to make a dictionary from keyword arguments, introduced in Chapter 4 and covered in Chapter 8)". The "dict()" in this should be fixed-width font.

It's impossible to add forward notes for every deferred topic like this, but this case seems especially glaring to me.

--------
This item is part of a batch that was cut-and-paste from the
following webpage, which may contain additional information
about the issue:

http://www.rmi.net/~lutz/lp4e-updates.html

[Nov-09-09] Page 185, *args in call forward reference (but noted earlier)

On this page, the syntax '...'.format(*parts) makes use of special call syntax to unpack an iterable's items into separate arguments. This feature isn't covered in full until much later in the book. This isn't a forward reference error, though -- although it's not noted here specifically, this syntax is noted briefly earlier in the book. For forward references like this, the book doesn't necessarily repeat the note at every occurrence.

To clarify further, though, add the following at the very end of the second paragraph on page 185 and just before the colon there: "(note the use of *parts here to unpack a tuple's items into individual arguments, as we did on Page 132 when studying fractions; more on this in Chapter 18)". The "*parts" in this should be fixed-width font.

Specifically, this call syntax is described in conjunction with fraction ratios on page 132, which states: "(the * in the second test is special syntax that expands a tuple into individual arguments; more on this when we study function argument passing in Chapter 18)". A **dict forward reference is also noted on page 189, which states: "(as we?ll see in Chapter 18, the **data in the method call here is special syntax that unpacks a dictionary of keys and values...)". Like many things in this book, full coverage is deferred until later in the book; more on this on the book's updatespage listed below.

--------
This item is part of a batch that was cut-and-paste from the
following webpage, which may contain additional information
about the issue:

http://www.rmi.net/~lutz/lp4e-updates.html

[Oct-28-09] Page 135, set items are immutable (though sets are not)

At the start of the 2nd paragraph under "Set literals in Python 3.0", the book states that "This syntax makes sense, given that sets are essentially like valueless dictionaries--because they are unordered, unique, and immutable, a set?s items behave much like a dictionary?s keys." One reader suggested changing "immutable" to "mutable" here, but that would break the intent. This description is technically correct as is, since its last half is discussing the items in a set rather than a set itself. Still, it's a bit ambiguous as worded.

For clarity, change the second part of this sentence as follows: "This syntax makes sense, given that sets are essentially like valueless dictionaries--because a set's items are unordered, unique, and immutable, the items behave much like a dictionary?s keys."

--------
This item is part of a batch that was cut-and-paste from the
following webpage, which may contain additional information
about the issue:

http://www.rmi.net/~lutz/lp4e-updates.html

[Nov-03-09] Page 123, three numeric tests: run under 2.X not 3.X

I ran the last code listing on page 123 (along with its continuation on the next page) under Python 2.6 on purpose, in order to show the long precision used for large integers. This should be obvious given the discussion of the "L" suffix for 2.6 longs earlier in the chapter. I didn't explicitly state that this example gives 2.6 output format, though, and that may confuse, given that other examples are either 3.X or labeled as 2.X.

To clarify, at the end of the last paragraph on page 123 and just before that page's last code listing, add "(run in 2.6 here to reveal the long precision)" just before the colon there.

For reference (not to be placed in the book), here's how this looks if run in 3.0 -- it uses long precision, but you can't really tell:

>>> X = 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFF
>>> X
5192296858534827628530496329220095
>>> oct(X)
'0o17777777777777777777777777777777777777'
>>> bin(X)
'0b11111111111111111111111111111111111111111111 ...and so on...

--------
This item is part of a batch that was cut-and-paste from the
following webpage, which may contain additional information
about the issue:

http://www.rmi.net/~lutz/lp4e-updates.html

[Nov-03-09] Page 98, binary file intro example: assumes a file exists

The last code listing on this page opens and reads a file named 'data.bin'. For the purposes of this introductory overview chapter on types, it assumes this file exists, but does not explicitly create it to avoid getting too deep into the 3.0 bytes topic (i.e., it's a deliberately partial example like much in this chapter). If you are working along with the examples here, though, note that this code (and its elaboration on page 234) won't work unless you've already created the binary data file in your working directory; to see how to do so, watch for the more in-depth files coverage of Chapter 9 and 36.

To minimize confusion, though, we'll add the following text at the very end of the last paragraph on page 98, just before the colon there: "(the following partial example assumes there is already a binary file in your current directory)". [Reprints: omit "partial example" if this doesn't fit.]

--------
This item is part of a batch that was cut-and-paste from the
following webpage, which may contain additional information
about the issue:

http://www.rmi.net/~lutz/lp4e-updates.html

[Nov-08-09] Page 91, mention the dict(name=val) form in intro chapter

On page 91, at the very end of the last paragraph before heading "Nesting Revisited", add the following sentence: "As we'll learn later, dictionaries may also be made by passing keyword arguments to the type name: dict(name='Bob', job='dev', age=40) makes the same dictionary.", where the "dict(...)" part in this is code in fixed-width font. [Reprints: it looks like there is space for the insert, but please ask me how to shorten if not.]

A related page 186 insert is described ahead. I'm adding this because the dict() call isn't formally described until the list/dict chapter on pages 208 and 217, but pops up in a few examples shortly before that--in some new str.format() calls on pages 186-189. In fact, I would mark this unnoted forward reference as an error (other forward references usually say they are), except that its usage is probably apparent to most readers; it does get covered just 20 pages later; and because of the general book-wide principles concerning forward references which are described in the general book notes section of my book updates webpage noted below. In short, some topics are postponed until later, largely due to Python's design.

--------
This item is part of a batch that was cut-and-paste from the
following webpage, which may contain additional information
about the issue:

http://www.rmi.net/~lutz/lp4e-updates.html

[Nov-03-09] Page 90, generators and iteration protocol: iter() optional

The first code listing on this page shows a generator expression, and a valid but intentionally abbreviated exploration of applying the iteration protocol to it. As suggested briefly later in this chapter and explained more completely later in the book, the full realization of the iteration protocol here would be: G = (...); I = iter(G); and next(I) until StopIteration is raised. Because generators are their own iterator (that is, iter(G) returns G itself), the code on this page is able to skip the iter(G) call and proceed to next(G). This is all explained in-depth later in the book, and this chapter is explicitly designed to be an incomplete first look (like much in this chapter, the full story on iteration is far too subtle for a first pass over built-in types).

Still, this example might seem to contradict the iterator protocol description later in this chapter on first glance. To avoid some confusion, on the second line of page 90 (the code line that reads ">>> next(G)"), add a comment near the end of this line that reads "# iter(G) not required here", where the "#" in this lines up vertically with the "#" of the preceding line. That is, the first two lines on this page should look like:

>>> G = (sum(row) for row in M) # Create a generator of row sums
>>> next(G) # iter(G) not required here

--------
This item is part of a batch that was cut-and-paste from the
following webpage, which may contain additional information
about the issue:

http://www.rmi.net/~lutz/lp4e-updates.html

[Oct-28-09] Page 86, mention XML parsing in types intro chapter 4

In paragraph 3 on this page, I want to mention XML parsing at the end of the strings introduction. Change "even more advanced language processing, including natural language processing." to read "even more advanced text and language processing, including XML parsing and natural language analysis.".

XML parsing is mentioned in the intro section of the in-depth string chapter 7 along with a forward link to XML examples in Chapter 36, so this is perhaps nitpicking; an XML teaser here is warranted, though.

--------
This item is part of a batch that was cut-and-paste from the
following webpage, which may contain additional information
about the issue:

http://www.rmi.net/~lutz/lp4e-updates.html