Errata

print [v*10 for v in l1 if if v1>4]

should read

print [v*10 for v in l1 if if v>4]

The example list is given as:

string_list = ['a', 'b', 'c', 'd',]

The third example under the List and dictionary heading states the following:

string_list[2] # returns 'b'

Actually, since [2] is the offset, and list offsets start with [0],

string_list[2] should actually return 'c' not 'b' .

v1 should be v

In the addlinkref function, the call to the separatewords function appears as separateWords rather than in all lowercase, as it defined within the text of the book. Mixing the two 'spellings' causes an error.

In the example code, there's a bug in 'chapter6/feedfilter.py'. I think you should change the code like this.

Line 22:
-- print 'Guess: '+str(classifier.classify(entry))
++ print 'Guess: '+str(classifier.classify(fulltext))

Line 26:
-- classifier.train(entry,cl)
++ classifier.train(fulltext,cl)

(submitted by a customer via email)

print [v*10 for v in l1 if v1>4] should be
print [v*10 for v in l1 if v>4]

I the Programming Collective Intelligence Book, in the Preface in Python Tips section - there is a mistake:
----------------

Python has a good set of primitive types and two that are used heavily throughout this book are list and dictionary. A list is an ordered list of any type of value, and it is constructed with square brackets:

number_list=[1,2,3,4]

string_list=['a', 'b', 'c', 'd']

mixed_list=['a', 3, 'c', 8]

A dictionary is an unordered set of key/value pairs, similar to a hash map in other languages. It is constructed with curly braces:

ages={'John':24,'Sarah':28,'Mike':31}

The elements of lists and dictionaries can be accessed using square brackets after the list name:

string_list[2] # returns 'b'

ages['Sarah'] # returns 28

----------------
In reality it will return c , as in python an array starts from 0.

Have a nice day. ?
Best regards,
PhD Student Pavel Kucherbaev
University of Trento, Italy

Error was found in Kindle edition, copyright 2008

The Dupree data point in Figure 2-3 is incorrect. It is shown at [1,2] in the publication; however, according to the values in the "critics" dictionary (both in the book and online), it should be displayed at [3.5, 2.5].

The default value for sigma is printed as 10.0, but this should be 1.0 in order to match the graph and the printed results.

Wrong:

def gaussian(dist,sigma=10.0):
return math.e**(-dist**2/(2*sigma**2))

Right:

def gaussian(dist,sigma=1.0):
return math.e**(-dist**2/(2*sigma**2))

The input vector in the example below is wacky:

>>> numpredict.cumulativegraph(data, (1,1), 120)

To match the graph in Figure 8-10, try the following:

>>> numpredict.cumulativegraph(data,[99,20],120)

(Page 186 in printed version?)

The input vector and the high value are wacky:

>>> numpredict.probabilitygraph(data,(1,1),6)

To match Figure 8-11, try the following:

>>> numpredict.probabilitygraph(data,[99,20],120)

(Page 186?)

In addition to my previous submission, the fabs function does not need to be used since the difference is being squared, so my suggested edit:

sqrt(pow(4.5-4,2)+pow(1-2,2))

reload(recommendations)

is performed before the recommendations module has ever been loaded.

Book gives sim_distance between Lisa and Gene of 0.148148148148. My result was 0.29429805508554946; verified with calculator.

last line of code reads:

return 1/(1+*sqr*t(sum_of_squares))

Should read

return 1/(1+sqrt(sum_of_squares))

AND

Result is given below as 0.148148148148

This is equivalent to 1/(1+5.75)

5.75 is sum_of_squares NOT sqrt(sum_of_squares)

Result should be 0.294298055086

This is (supposedly) the result of using the function 'sim_distance' as it appears on this page, but it actually uses the statement

return 1/(1 + sum_of_squares),

rather than the (correct) statement

return 1/(1 + sqrt(sum_of_squares)).

Apparently, the error is the result of *using* the wrong version of the function, while the correct version is *printed* .

The sim_distance function should return 1/(1+sqrt(sum_of_squares)) rather than 1/(1+sum_of_squares) when inverting the Euclidean distance. The formula for Euclidean distance includes a square root, but the square root is never taken of the sum of all the squares in this segment of code.

In the code snippet for sim_distance:

return 1/(1+sqrt(sum_of_squares))

Shouldn't this be

return 1/(1+sqrt(sum_of_squares/len(si)))

Also, in the example code zip on the companion website, this is

return 1/(1+sum_of_squares)

which should also be

return 1/(1+sqrt(sum_of_squares/len(si)))


I guess this error has no effect when getting recommendations for a single movie or critic; the resulting similarity values deviate, but keep the same order. However, this error has serious effect when ordering similarities over movies/critics. This is done in getting recommendations item based, on page 24.
Am I correct? (If I'm not, please excuse me for this sunday morning observation *yawn* )

The correct similarity score is 0,294298055085 instead of 0.148148148148.

The numerical output values of some examples, in chapter 2, including the above seems to be due to an error in the Examples (PCI_Folder), chapter2, recommendations.py, line 38.
Wrong: return 1/(1+sum_of_squares)
Right : return 1/(1+sqrt(sum_of_squares))

# One would have had to execute and 'import' prior to a 'reload' ...

>>> import recommendations
>>> reload(recommendations)

>>> recommendations.sim_distance(recommendations.critics, 'Lisa Rose', ... 'Gene Seymour')
0.148148148148

should be:

>>> recommendations.sim_distance(recommendations.critics, 'Lisa Rose', ... 'Gene Seymour')
0.294298055085

The output is wrong, but the sim_distance function is actually correct. My guess is the author forgot to update his code to reflect the code in the book. This error is also present in the downloadable version of the source code.

It can easily be fixed by replacing the return statement in the sim_distance function in recommendations.py.

Replace:

return 1/(1+sum_of_squares)

with:

return 1/(1+sqrt(sum_of_squares))

recommendations.sim_distance(recommendations.critics, 'Lisa Rose', 'Gene Seymour') results in a similarity score of: 0.29429805508554946.

>>> 1/(1+sqrt(5.75)) #where 5.75 is the 'sum_of_squares'
0.29429805508554946

Not taking the square root of the 'sum_of_squares' will result in the value printed in the book:
>>> 1/(1+5.75)
0.14814814814814814

Earlier on in the chapter, the Euclidian Distance score is defined as:
1/(1+sqrt(sum_of_squares))

However, the code example on page 11 leaves out the square root.

---- snip -----
from math import sqrt
# Returns a distance-based similarity score for person1 and person2

def sim_distance(prefs,person1,person2):
# Get the list of shared_items
si={}
for item in prefs[person1]:
if item in prefs[person2]:
si[item]=1
# if they have no ratings in common, return 0
if len(si)==0:
return 0
# Add up the squares of all the differences
sum_of_squares=sum([pow(prefs[person1][item]-prefs[person2][item],2)
for item in prefs[person1] if item in prefs[person2]])
return 1/(1+sum_of_squares)
---- end snip ----

The last line of this snippet should be:
return 1/(1 + sqrt(sum_of_squares))

In addition, the result in the interpreter output for the code example contains the incorrect value, since the formula was wrong.

By the way, great book so far, I'm really enjoying it.

The square root is not applied in the sim_distance function as is described on the previous page (page 10) in the section titled Euclidean Distance Score.

E.g.
The last line of the function is:
return 1/(1+sum_of_squares)

but possibly could/should be:
return 1/(1+sqrt(sum_of_squares))

If this is an error/omission it also follows through to the example usage of the function, also on page 11 directly beneath the function code and the example code download accompanying the book.

If it is not an error/omission then I think an explanation should be included explaining why the square root is not applied; As it appears to contradict the explanation of how to calculate Euclidean Distance on page 10.

It says:

"return 1/(1+sqrt(sum_of_squares))"

If you run this, you get a different result than stated in the book. The downloaded book code samples say:

"return 1/(1+sum_of_squares)"

Which gives the proper results, and the right results as shown in the book.

sum_of_squares in downloadable code (correct)

sum_of_squares=sum([pow(prefs[person1][item]-prefs[person2][item],2)
for item in prefs[person1] if item in prefs[person2]])



sum_of_squares in printed code (incorrect)

sum_of_squares = sum([pow(prefs[person1][item]-prefs[person2][item],2)
for item in si])

sum_of_squares in downloadable code-- yields (0.14814814814814814)

sum_of_squares=sum([pow(prefs[person1][item]-prefs[person2][item],2)
for item in prefs[person1] if item in prefs[person2]])

return 1/(1+sum_squares)

sum_of_squares in printed code -- yields (0.29429805508554946)

sum_of_squares = sum([pow(prefs[person1][item]-prefs[person2][item],2)
for item in si])

return 1/(1+sqrt(sum_squares))

The similarity between Lisa Rose and Gene Seymour should be 0.294298055086 instead of 0.148148148148.

In Figure 2-3 on page 12, Dupree is plotted with a score of 1 from Jack Matthews, which doesn't match the score specified earlier (3.5).

Definition of sim_pearson is subject to integer math errors, at least in Python 2.5.1. The number of overlapping elements is saved as "n = len(is)". This is used later to determine the numerator of the formula.

It turns out this can allow the function to return values > 1, for this reason:

13 * 11 / 3 ==> 47

whereas:

13 * 11 / 3.0 ==> 47.666666666666664


Changing the initial line to be "n = float(len(si))" is enough to prevent this.


As someone in a separate errata noted, the function should also return 1.0 if the denominator is 0.

As person is not in the list, the second condition in the original code will never be fulfilled:

# only score movies I haven't seen yet
if item not in prefs[person] or prefs[person][item]==0:

The code may ignore the zero scored items by the "other" person using this code:

# only score movies I haven't seen yet that contribute to the recommendation
if item not in prefs[person] or prefs[other][item]!=0.0:

In the pydelicious.py module, an exception is raised when "feedparser" is imported.

New del.icio.us API uses "url" field instead of "href" field. All the text referring to "href" field should be updated.

Upon running the install script for version 0.6, I get 'ImportError: No module named etree.ElementTree'

How do you get the del.icio.us API to work??? There is absolutely no documentation for it. The documentation folder is empty and there is no documentation on the code download site. It doesn't even say which version of Python is needed! You should add the version of the API that works for you.

"To make things even easier for you, there is a Python API that you can down-load from http://code.google.com/p/pydelicious/source or http://oreilly.com/catalog/9780596529321."

I clicked on the book link to oreilly.com but did not find the API code there. I downloaded the book code and did not find it there either!

The noted mistake on pg 11 with regards to calculating the similarity based on the distance metric propagates to this code sample as well. The numbers are slightly incorrect. They should really be:

[(3.1667425234070894, 'The Night Listener'),
(2.936629402844435, 'Just My Luck'),
(2.868767392626467, 'Lady in the Water')]

The code to split text into words uses this regular expression:

[^A-Z^a-z]

the above includes caret ('^') in the set with the alphabetic characters, so caret will not be used to split words. For example:

>>> re.compile("[^A-Z^a-z]").split("foo^bar")
['foo^bar']


The correct regex is [^A-Za-z]:

>>> re.compile("[^A-Za-z]").split("foo^bar")
['foo', 'bar']

In the first for loop of the first code snippet:

Instead of:
for e in d.entries:
if 'summary' in e: summary=e.summary

The return d.feed.title,wc will throw an AttributeError if any of the feed urls in feedlist.txt no longer exist.

This can be catered for by catching the exception as follows:

try:
return d.feed.title,wc
except AttributeError:
return "nofeed",wc

1. In the main loop the first line with function call

title,wc=getwordcounts(feedurl)

generates an uncaught exception:

TypeError: 'NoneType' object is not iterable


Adding the try..except fixes the problem:

2. In the second code listing feedlist variable is not defined

feedlist.add(feedurl)

add method doesn't exist. should probably read feedlist.append(feedurl)

blog = blog.encode(`ascii`, `ignore)

it's not single quotes, and the terminating quote on ignore is missing.

The apcount map is supposed to contain "..the number of blogs each word appeared in", yet the code contains the following statement:

for word,count in wc.items():
apcount.setdefault(word,0)
if (count>1):
apcount[word]+=1

in other words, apcount actually contains the number of blogs each word appeared in _more than once_. This is probably not what the author wants, and I propose the following change to the last two lines of code:

if (count>0):
apcount[word]+=1

The Pearson correlation coefficient error that anonymous first identified on p. 13 also appears here. The line reading,

num=pSum-(sum1*sum2/len(v1))

should read

num=pSum-(sum1*sum2/float(len(v1)))

to avoid integer roundoff errors.

This Pearson correlation coefficient function contains another slight error:


den=sqrt((sum1Sq-pow(sum1,2)/len(v1))*(sum2Sq-pow(sum2,2)/len(v1)))


should read


den=sqrt((sum1Sq-pow(sum1,2)/float(len(v1)))*(sum2Sq-pow(sum2,2)/float(len(v1))))


to avoid integer roundoff errors.

[rownames[r] for r in k[0]]

should read,

[blognames[r] for r in kclust[0]]


Likewise,

[rownames[r] for r in k[1]]

should read,

[blognames[r] for r in kclust[1]]

A try catch block can be put in the code, as when the program fails to load data for a user posts will become undefined in the ensuing portion of code.

def fillItems(user_dict):
all_items={}
# Find links posted by all users
for user in user_dict:
print user;
for i in range(3):
try:
posts=get_userposts(user)
break
except:
print "Failed user "+user+", retrying"
time.sleep(4)
try: // put this try-catch block additionally
for post in posts:
url=post['href']
user_dict[user][url]=1.0
all_items[url]=1
except:
print 'no posts'

In this line:
"Once this is done, the code first has to create a list of items than more than five people want, ..."
change "five" to "ten". Alternatively, change 10 to 5 in the code listing following this paragraph.

The variable "outersum" is defined but never used.

When using a query not in the database I receive the following error

>>> e.query('gorgonzola')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "searchengine2.py", line 223, in query
rows,wordids=self.getmatchrows(q)
File "searchengine2.py", line 201, in getmatchrows
cur=self.con.execute(fullquery)
pysqlite2.dbapi2.OperationalError: near "where": syntax error

I intend to find a solution.

dist = sum([abs(rows[i] - row[i-1) for row in range(2, len(ow))])

This makes no sense. Presumably sum should be min.

This is a lovely book, but I agree with an earlier submitter, very poorly proofread. This is especially the case as the code isn't commented. Even if the code were error free, it is quite terse and hard for a non Python expert to read. The errors really detract from readability.

Page C has only three links to other pages. It should have four links to other pages.

Figure 4-3. Calculating the PageRank of A
Page C links to A and four other pages the text says. This is confirmed by the calculation for PR(A) on page 71 where PR(C)/links(C) is defined as 0.7/5.
The figure, however, only displays C's link to A and THREE other pages, which is correct for B, not C.

The outgoing links (arrows) from page C are wrong. In the text you read that "... and C links to four other pages.". Thus, the arrows going out in the drawing from page C should be 4 and not 3. This is also confirmed by the formula to compute PR(A) few lines below, where for page C you can read: PR(C)/links(C) and this is 0.7/5.
Thanks,
Christian

To be most effective, a call to self.dbcommit() should be inserted after the self.con.execute('drop table if exists pagerank') call (i.e. just after the first command of the calculatepagerank function). This permits calculatepagerank to be called multiple times in a single python session without the database returning an error stating that it already contains a pagerank table, and subsequently remaining locked due to the error.

the statement:
for i in range(3): print cur.next()
does nothing, why have a for loop if you are not going to use the index?

Actually this can be fixed, and there are a lot of other typos in this book, to anyone looking for solutions here is what I did, I hope this saves you some time
1. look at the example code available to download, it is sometimes different from the code in the book
2. check here http://blog.kiwitobes.com/?p=44
to see if anyone else had the same problem as you

The list variable ai is a list of 1's. The value of ai never changes and it is only used to multiply against another list. That multiplication is redundant as it just results in the original list.

Also, self.ai is created twice. Once in the middle of setupnetwork and at the start of feedforward. It is never changed and should result in the same value since setupnetwork is always run right before feedforward.

For the output of .getresult() (which comes from .feedforward()), it states:

[0.76,0.76,0.76]

However, I got:

[0.076,0.076,0.076]

Since these values are produced by TANH and the values are off by exactly one order-of-magnitude, I find it unlikely to be a decimal problem somewhere in the code, and more likely that it's a simple misprint.

The paragraph references a link to retrieve the sample data:

http://kiwitobes.com/optimize/schedule.txt

But this is a dead link. It looks like all the data (for this and other chapters) has been moved to the sample code zip file but there's no reference to this on the support page for the book.

for d in range(0, len(r), 2):
should be
for d in range(0, len(people)):
Iterating over the schedule is incorrect, the list of people should be iterated over. This is why the out and ret variables are retrieved via r[2*d] and r[2*d+1], respectively. As is, this causes an index out of bounds exception since the length of the schedule is twice the length of the number of people.

The index for the line beginning with out= should be [int(r[2*d])] rather than [r[d]], and the index for the line beginning with ret= should be [int(r[2*d+1])] rather than [r[d+1]].

There is no need to change the range of the for loop in the printschedule function, however, or to reverse any of the destination and origin assignments, as was erroneously reported elsewhere in this errata list.

In this case, each number can represent which flight a person chooses to take, where 0 is the first flight of the day, 1 is the second, and so on. Since each person needs an outbound flight and a return flight, the length of this list is twice the number of people.
For example, the list:
[1,4,3,2,7,3,6,3,2,4,5,3]

Represents a solution in which Seymour takes the second flight of the day from Boston to New York, and the fifth flight back to Boston on the day he returns. Franny takes the fourth flight from Dallas to New York, and the third flight back.


Should be :
Represents a solution in which Seymour takes the second flight of the day from Boston to New York, and the (seventh) flight back to Boston on the day he returns. Franny takes the (fifth) flight from Dallas to New York, and the (fourth) flight back.

int(sol[d]) should be int(sol[2*d])
in(sol[d+1]) should be int(sol[2*d+1])

and therefore the calculate result at the next page should be 5469

In theory, you could try every possible combination, but in this example there are 10 flights, all with 6 possibilities, giving a total of 6^10 combinations.

instead of

In theory, you could try every possible combination, but in this example there are 16 flights, all with 9 possibilities, giving a total of 9^16 combinations.

b'cause 6 family members having 10 flights to LGA

I believe that calculation should be:

(10 flights to LGA * 10 flights from LGA) ^ (# of family members) =
(10^2)^6 = 10^12

or in Python,
(10**2)**6 = 10**12

Oh, and 9**16 is much closer to 300 trillion than it is to 300 billion

The randomoptimize function should return bestr, not r.

The domain goes from 0-10, i.e. 0..9 rather than 0..7. (See schedule.txt for confirmation; also on p. 97, 8 could not be included as part of the solution if the highest available value were 7.) This means the line specifying the domain should read:

domain=[(0,10)]*(len(optimization.people)*2)

The spelling of neighbors is inconsistent in the code sample. All instances of the word neighbors/neighbours in the code sample should be changed either to "neighbors" or "neighbours", but not a mix of both.

The probability of a higher-cost solution being accepted should read:

p = e^(-(highcost-lowcost)/temperature)

The mutate function is missing an else clause (or some other mechanism for returning a default value). A possible solution would be to remove the elif conditional and guarantee that the function always returns a value:

# Mutate operation
def mutate(vec):
i=random.randint(0,len(domain)-1)
if random.random()<0.5 and vec[i]>domain[i][0]:
return vec[0:i]+[vec[i]-step]+vec[i+1:]
else:
return vec[0:i]+[vec[i]+step]+vec[i+1:]

the mutate function should has a else clause which return vec. Otherwise None will be added into pop occasionally

Firstly in this function correct the mistyped 'mutprod' to 'mutprob'

The mutate function goes out of bounds for different step values plus the previously reported error so here is my corrected version

def mutate(vec):
i=random.randint(0,len(domain)-1)
# Corrected so different step values can be added
if random.random()<0.5 and vec[i]-step>domain[i][0]:
return vec[0:i]+[vec[i]-step]+vec[i+1:]
elif vec[i]+step<domain[i][1]:
return vec[0:i]+[vec[i]+step]+vec[i+1:]
else:
return vec

Also during the mainloop mutations/crossovers are made but not scored, so my solution is to make maxiter +1 and in the last iteration just score it without adding mutations;

# Main loop
for i in range(maxiter):
scores=[(costf(v),v) for v in pop]
scores.sort()
if i == maxiter-1: pass # Corrected from here for last iteration
else:
ranked=[v for (s,v) in scores]

# Start with the pure winers
pop=ranked[0:topelite]

# Add mutated and bred forms of the winners
while len(pop)<popsize:
if random.random()<mutprob:

# Mutation
c=random.randint(0,topelite)
pop.append(mutate(ranked[c]))
else:

# Crossover
c1=random.randint(0,topelite)
c2=random.randint(0,topelite)
pop.append(crossover(ranked[c1],ranked[c2]))

# Print current best score
print scores[0][0]

return scores[0][1]

The output of dorm.printsolution(s) uses the value of s as generated by s=optimize.randomoptimize(), for a solution cost of 18 (the original generated solution.)

optimize.geneticoptimize() is then called, but the results are not reassigned to s, and the output is confusing.

It might be more clear if this session were changed to:

reload(dorm)
s=optimize.randomoptimize(dorm.domain,dorm.dormcost)
dorm.printsolution(s)
dorm.dormcost(s)
r=optimize.geneticoptimize(dorm.domain,dorm.dormcost)
dorm.printsolution(r)


so that the before and after can be seen.

The PIL wasn't showing me any results so I print it out as a a jpg instead

from PIL import Image, ImageDraw

def drawnetwork(sol):
jpeg='network_diagram.jpg'
# Create the image
img=Image.new('RGB',(400,400),(255,255,255))
draw=ImageDraw.Draw(img)

# Create the position dict
pos=dict([(people[i],(sol[i*2],sol[i*2+1])) for i in range(0,len(people))])

# Draw links
for (a,b) in links:
draw.line((pos[a],pos[b]),fill=(255,0,0))

# draw people
for n,p in pos.items():
draw.text(p,n,(0,0,0))
img.save(jpeg,'JPEG')

After importing docclass, the db should be initialized.

cl.setdb('test.db')

Or this line can be added to docclass.py

"In this case, you can calculate
the probability that a word is in a particular category by dividing the number of times
the word appears in a document in that category by the total number of documents
in that category."

in this sentence, it indicated that the
Probability of a word = total number times the word appears / total count of document
But I think it is wrong , right one must as follow two case:

A. Probability of a word = total number times the word appears/ total count of the word in all document

B. Probability of a word = total number times the word appears / total count of the word in the document.

In the entryfeatures function, summarywords is produced by calling the lower() method on each string produced by splitting the summary text. The function later tries to count uppercase words by calling the isupper() method on each string in summarywords. Since these strings have already been converted to lower case, isupper() will never return true.

I found a minor issue with the code example at page 137. The code tries to detect overuse o uppercase use but it fails because it converts all the words to lowercase in the second list comprehension of the function.

summarywords = [s.lower() for s in splitter.split(entry['summary'])
if len(s) > 2 and len(s) < 20]

The code to detect the uppercase abuse correctly is (see that I called lower() in f[w.lower()] and in f[towowords.lower()]:

def entryfeatures(entry):
splitter = re.compile('\\W*')
f = {}

# Extract the title words and annotate
titlewords = [s.lower() for s in splitter.split(entry['title'])
if len(s) > 2 and len(s) < 20]

# Extract the summary words
summarywords = [s for s in splitter.split(entry['summary'])
if len(s) > 2 and len(s) < 20]

# Count uppercase words
uc = 0
for i in range(len(summarywords)):
w = summarywords[i]

if (w.isupper()):
uc += 1

f[w.lower()] = 1

# Get word pairs in summary as features
if i < len(summarywords) - 1:
twowords = ' ' . join(summarywords[i:i+1])
f[twowords.lower()] = 1

# Keep creator and publisher whole
f['Publisher:'+ entry['publisher']] = 1

# UPPERCASE is a virtual word flagging too much shouting
if (float(uc) / len(summarywords) > 0.3):
f['UPPERCASE'] = 1

return f

Importing decision_tree_example.txt keeps "\n"s in the data,
so remove them or this taints the data
example from uniquecounts() {'Basic\n': 4, 'Premium\n': 1, 'Basic': 1}

This is how I felt like doing it today

my_data=[line.split('\t') for line in file('decision_tree_example.txt')]
for a in range(len(my_data)):
for b in range(len(my_data[a])):
my_data[a][b] = my_data[a][b].replace('\n','')

divideset() was splitting the numerical-data as string-data because that was is it was stored in my_data.
So, instead of splitting >=value it was splitting ==value
The corrected version produces the same tree, but with different datasets who knows?

def divideset(rows,column,value):
try: value = int(value) # Convert to int here
except: pass

split_function=None
if isinstance(value,int) or isinstance(value,float):
split_function=lambda row: int(row[column])>=value # Convert here too
else:
split_function=lambda row:row[column]==value

etc......................

The recursive calls at the end of the buildtree function should propagate the scoref parameter. Otherwise if you use a scoref function besides the default "entropy" function it will only be used on the first call.

So instead of

trueBranch=buildtree(best_sets[0])

it should be

trueBranch=buildtree(best_sets[0], scoref)

The line

delta=entropy(tb+fb)-(entropy(tb)+entropy(fb)/2)

should be

delta=entropy(tb+fb)-(entropy(tb)+entropy(fb))/2

mdclassify function is called qualified by treepredict2, but treepredict2 is neither imported (doesn't exist anyway) or defined anywhere. Changing

treepredict2.mdclassify(['google', 'France',None,None],tree)

to simply

treepredict.mdclassify(['google', 'France',None,None],tree)

appears to generate the intended results.

The reported results for the first test of the mdclassify function are different than those produced by the code in the downloadable code samples. Here's what I get with Python 2.6.4:

>>> tree = treepredict.buildtree(treepredict.my_data)
>>> treepredict.mdclassify(['google', None, 'yes', None], tree)
{'Premium': 2.25, 'Basic': 0.25}

It appears as though the Zillow API does not return "totalRooms" anymore (assuming it once did). Furthermore, some of the houses that it searches for appear to return no actual values from Zillow, or still cause the exception block to be hit. Later on, when the code asks for len(row) in the variance method of treepredict.py, this will cause "TypeError: object of type 'NoneType' has no len()".

As a work around:

rooms=doc.getElementsByTagName('totalRooms')[0].firstChild.data

should be commented out. Also, in getpricelist() function, change

l1.append(data)

to

if data != None
l1.append(data)


this should prevent null rows getting added and therefore prevent the TypeError exception described above.

The pdf says to call geneticoptimize using;
optimization.geneticoptimize(numpredict.weightdomain,costf,popsize=5,lrate=1,maxv=4,iters=20)

I assume this was an different version of geneticoptimize(), as the values accepted by geneticoptimize are
geneticoptimize(domain,costf,popsize=50,step=1,mutprob=0.2,elite=0.2,maxiter=100)

I tried this instead, guessing that 'maxv' related to 'elite'.
optimization.geneticoptimize(numpredict.weightdomain,costf,popsize=5,step=1,elite=0.2,maxiter=20)

the cumulativegraph function is being passed the vector (1,1), which means it will draw the cumulative probability for the price of bottles of wine which have a rating of 1 and are 1 year old. These wine bottles would be so terrible that the cumulative probability would reach 1 at a price of 0, meaning there's nothing to see on the graph generated.

A better example would be to use say a wine bottle rating of 99 that's 10 years old, with a call like:

numpredict.cumulativegraph(data,(99,10),120)

This way, they'll actually be something useful displayed on the graph when its printed and it should look more like Figure 8-10 does.

Like the error on page 186, using the vector (1,1) - a wine bottle with a rating of 1 and 1 year old, means that the wine would be so bad that it would all cost essentially 0. Instead, use a vector like say (99,10).

Also, the value of 6 used for high is quite small. To get something that looks similar to figure 8-11, the value 120 should be used instead. So, the line which reads:

numpredict.probabilitygraph(data,(1,1),6)

should become something like

numpredict.probabilitygraph(data,(99,10),120)

The paragraph after the code sample states "The points will be O if the people are a match and X if they are not." The code sample however draws a scatterplot where people that are a match are represented with a green O and people that are not a match are represented with a red O (not an X). To get a scatterplot that uses red X's instead of red O's, change the line:

plot(xdn,ydn,'ro')

to

plot(xdn,ydn,'rx')

Alternatively, to get a scatterplot that looks like that in figure 9-1 (which uses neither O's nor X's), change the same line:

plot(xdn,ydn,'ro')

to

plot(xdn,ydn,'r+')

If the plotagematches function is really to be called from one's Python session (rather than being added to the advancedclassify.py file) then one does not need to reload advancedclassify, and should only type "plotagematches(agesonly)" instead of "advancedclassify.plotagematches(agesonly)" into the Python session.

There is a sentence which states "There are two other points, X0 and X1, which are examples that have to be classified". The diagram on the other hand uses points X1 and X2, but no X0. I believe that the sentence should be changed to "There are two other points, X1 and X2, which are examples that have to be classified."

The URL for Yahoo! Maps Web Services has changed. The line reading,

'http:\\api.local.yahoo.com/MapServices/V1/'+\

should now read,

'http://local.yahooapis.com/MapsService/V1/'+\

Apparently there are two 824 3rd Avenues in New York city: one that's approximately 0.9 miles from 220 W 42nd St, and another that's about 6.6 miles away. It might be good to include zip codes with some, or all, of the addresses from the matchmaker.csv file to avoid confusion. [The new Yahoo! Maps Services returns the latitude and longitude for the second address - the one that's circa 6.6 miles away - as the default rather than those of the address that's approx. 0.9 miles away, as listed in the book.]

In the book, scaleinput should be defined as:

# Create a function that scales data
def scaleinput(d):
return [(d[i] - low[i]) / (high[i] - low[i]) for i in range(len(low))]

In the printed copy, the return statement tries to access d.data[i], even though the data member was passed in to scaleinput, in both scaledata and in the interpreter example below the code.

Just above the definition of the rbf function there should be an

import math

statement to ensure that the constant math.e is defined.

To be in accordance with the other gammas, and with the code listed in PCI_code.zip, the first line of the rbf function definition should read,

def rbf(v1, v2, gamma=10):

For the purposes of the book, I believe that the gamma parameter is always explicitly passed, however.

Before you can execute any of the statements, you first need to reload advancedclassify using:

reload(advancedclassify)

The book also never defines the variable "offset" which is being parsed into the nlclassify function. I'm not sure what a reasonable value for this would be.

Just after the missing reload(advancedclassify) statement (See Bryce's comment for more), one should execute

offset=advancedclassify.getoffset(agesonly)

To properly define the otherwise undefined offset. This should produce a value of some -0.0076450020098023288.

The lines reading,

m.save(test.model)
m=svm_model(test.model)

should read,

m.save('test.model')
m=svm_model('test.model')

Also, in order for the svmc.pyd file to work with your Python environment, it must be of the correct version. The svmc.pyd file included in PCI_code.zip is for Python 2.4, the one available from the LIBSVM website (as of November 3, 2009) is for Python 2.6. I used this website to get a version that works with Python 2.5: <http://www.cs.sunysb.edu/~algorith/implement/libsvm/distrib/>. [It's also possible to build the svmc.pyd with C.]

isinstance(t, node):

should read

hasattr(t, 'children'):

The former does not function correctly (under Python 2.5.2 at any rate).

The line

if isinstance(t1, node) and isinstance(t2, node):

should read

if hasattr(t1, 'children') and hasattr(t2, 'children'):

The former does not allow the program to converge to the correct solution. [The latter code snippet also appears in the book's source code as distributed in PCI_Code.zip.]

Function selectindex() can return value
that is greater
or equal to popsize.
This results an error in caller function.

As currently presented, the when there is a tie between players i and j it is as if player i lost. Presently the code reads,

elif winner==-1:
losses[i]+=1
losses[i]+=1
pass

However, it should read,

elif winner==-1:
losses[i]+=1
losses[j]+=1
pass

Applying this change (replacing the second losses[i] with losses[j]) will assist in evolving improved game-playing AI. [The above error appears in the online code, PCI_code.zip, as well.]

The command should be
$ tar xvf numpy-1.0.2.tar
instead of
$ tar xvf numpy-1.0.2.tar.gz

The Pearson correlation coefficient error that anonymous first identified on p. 13 also appears here. The line reading,

n=len(x)

should read

n=float(len(x))

to avoid integer roundoff errors in a subsequent division [i.e. in num=pSum-(sumx*sumy/n)].