Errata

In the Integer Indexes section of Chapter 5 the following paragraph is ambiguous:

"To keep things consistent, if you have an axis index containing integers, data selection will always be label-oriented. For more precise handling, use loc (for labels) or iloc (for integers):"

To test this out I defined the following object:


`ser3 = Series(np.arange(4.), index=['a', 'b', -1, 34])`

and ran these two commands, both of which return 2.0:

`ser3[-1]`
`ser3[-2]`

`ser3.index` gives me "Index(['a', 'b', -1, 34], dtype='object')"

So, I think you could argue that the way Pandas actually works has some ambiguity to it and that the way the book describes it is the way it SHOULD work. But to describe the actual way this part of Pandas works, the following paragraph would be more accurate:

"To keep things consistent, if you have an axis index containing exclusively integers (mixed indexes will match on label first, and fall back to positional indexing), data selection will always be label-oriented. For more precise handling, use loc (for labels) or iloc (for integers):"

Or something of that nature.

Note from the Author or Editor:
will improve

Found this error on the kindle version, location 1741.

the line: In[76]: seq[3:4] = [6,3]

should be: In[76]: seq[3:4] = [6]

Note from the Author or Editor:
Confirmed. Will fix

table 2-5: detetime 格式化详细说明
%F %Y-%m-%d的简写（例如 2012-4-18）
actually 2012-04-18 %m:Double digit months

Note from the Author or Editor:
fixing in text

In Note 2, Ch. 1. Installing Necessary Packages:

Installing packages into conda environment uses activate instead of install:

Should be
conda install lxml beautifulsoup4 html5lib tables openpyxl /
requests sqlalchemy seaborn scipy statsmodels /
patsy sklearn

`df.iloc[where]` Selects single row or subset of rows from the DataFrame by label.

Should probably be "...from the DataFrame by *integer position*."

It looks like `.sort_values(by=...) is deprecated.

In [203]: frame.sort_index(by='b')
FutureWarning: by argument to sort_index is deprecated, pls use .sort_values(by=...)

In [207]: frame.sort_index(by=['a','b'])
FutureWarning: by argument to sort_index is deprecated, pls use .sort_values(by=...)

In [205]: frame.sort_values(by='b')

fixed the problem.

In [211]: pd.__version__
Out[211]: '0.19.2'

The input variable df is:

In [187]: df
Out[187]:
one two
a 1.40 NaN
b 7.10 -4.5
c NaN NaN
d 0.75 -1.3

The code in the book gives this result:

In [204]: df.sum(axis=1)
Out[204]:
a 1.40
b 2.60
c 0.00
d -0.55
dtype: float64

but shouldn't row "c" be "NaN" since we're summing together two NaNs? Here's what I get from my interpreter:

In [186]: df.sum(axis=1)
Out[186]:
a 1.40
b 2.60
c NaN
d -0.55
dtype: float64

In [188]: pd.__version__
Out[188]: '0.19.2'

The line of code:

data[-(names == 'Bob')]

Gives the deprecation warning:

DeprecationWarning: numpy boolean negative, the `-` operator, is deprecated, use the `~` operator or the logical_not function instead.

using numpy version 1.12.0

Using the tilde operator, as recommended silences the warning.

Text states: To exit the shell, press Ctrl-D or type the command exit() and press return.

On Windows, Ctrl-Z should be used.

Magic functions can be used by default without the percent sign ...

Some magic functions behave like Pyton functions and their output can be assigned to a variable:

In [22]: %pwd
Out [22]: '/home/west/code/pydata-book/

In [23]: foo = %pwd

----------------------------------------------------------------------------------

First, a single quote is missing from Out[22]

With ipython 6.3.1, although In [22] works using pwd without the leading percent sign, In [23] fails with "NameError: name 'pwd' is not defined"

Note from the Author or Editor:
Fixing this typo

I previously reported this issue, but it's a problem beyond the typo that was addressed in the reply.

"Magic functions can be used by default without the percent sign..."

This is not completely true.
For example, this variation of In[23] will not work:
foo = pwd

The % in front of the magic command can be skipped (by default) if the command is the first "word" on an IPython line. I have found that leading whitespace is not a problem.

Note from the Author or Editor:
Thanks. I will clarify that this is the only scenario where the % can be omitted

Running the matplotlib code exactly as printed inside Figure 2-6 gives a Type error:

TypeError: float() argument must be a string or a number, not 'builtin_function_or_method'

Note from the Author or Editor:
Thank you. Will fix

inconsistent description of a <= b, a < b (compared to the next line), should be for a < b, a <= b

Note from the Author or Editor:
Confirmed, will fix (in 3rd edition)

"Integer division not resulting in a whole number will always yield a floating-point number."

Actually, this is true for whole numbers too:

In [1]: 4/2
Out [1] : 2.0

Note from the Author or Editor:
Quite right. Will fix (in the 3rd edition)

Two illegal print statements:

print('It's negative')

Since the strings contain single quote characters, they should be delimited by double quotes.

Note from the Author or Editor:
Thank you, fixed

Although it's never explicitly described as such, the output of the dictionary on this page is showing the dictionary as unordered. This would be incorrect, as the book is utilizing Python 3.6, the version in which dictionaries changed to insertion-ordered.

Note from the Author or Editor:
Quite right. I'll correct this with the 3rd edition changes

Hello my friend.
This is related to usage and definition
of a set with reference to the 'set' function
in python.
While you may or may not agree whether this
is a minor technical mistake, it is a mistake in
terms of accuracy/precision. While a set is an
unordered collection of unique elements,
set as defined in python seems to be a 'sorted unordered
collection of unique elements.'
Thus, depending on input for set(), the actual output would
vary under those definitions -- subtle as they might be.

Note from the Author or Editor:
I will clarify.

The alternative syntax for a.issubset (b) and a.issuperset (b) shoule be "<=" and "=>" respectively (not N/A).

Note from the Author or Editor:
Fixing this.

"Like dicts, set elements generally must be immutable." should be "Like dict keys, set elements generally must be immutable."

Note from the Author or Editor:
Confirmed, will fix (in 3rd edition). Thank you

1st release print copy says, “...the result of fancy indexing is always one-dimensional.” However, there are example outputs in this section with more than one dimension. Is that because some of the examples in the section are not fancy indexing? If that’s the case, it’s unclear where the section is building up to a fancy indexing example as opposed to every example being fancy indexing. The number of dimensions in the output seems to be the number of array dimensions plus one minus the number of dimensions indexed.

Note from the Author or Editor:
The text is unclear, I will clarify

The book states when you are only passing a dict, the index in the resulting Series will have the dict's key in sorted order. However, this is not always the case. Running the code on my system I have the output pasted below. Looking at the output we see that returned series is not in sorted order.

sdata={'Ohio':35000,'Texas':71000,'Oregon':16000,'Utah':5000}

obj3=pd.Series(sdata)

obj3
Out[49]:
Ohio 35000
Texas 71000
Oregon 16000
Utah 5000
dtype: int64

Note from the Author or Editor:
Correct, pandas now obeys the insertion order of the dict. I will fix in the 3rd edition

The text says "When you are only passing a dict, the resulting Series will have the dict's keys in sorted order".

This doesn't appear to be true, either with the example given in the book, or with a repro (which proves the example is not the error). These keys seem _un_sorted to me when only passing in a dict.

>>> import pandas as pd
>>> sdata = { 'Ohio':35000, 'Texas':71000, 'Oregon':16000,'Utah': 5000 }
>>> obj3 = pd.Series(sdata)
>>> obj3
Ohio 35000
Texas 71000
Oregon 16000
Utah 5000
dtype: int64
>>> obj3.index
Index(['Ohio', 'Texas', 'Oregon', 'Utah'], dtype='object')

Note from the Author or Editor:
Confirmed that pandas now respects the "insertion order" of keys when creating a Series from a dictionary. Will fix in the 3rd edition

As presented, this line leads to "FutureWarning: Passing list-likes to .loc with missing label will raise KeyError in future." Revise to avoid warning.

Note from the Author or Editor:
Confirmed, will fix

"To keep things consistent, if you have an axis index containing integers, data selection will always be label-oriented."

I think this should be integer-oriented.

Note from the Author or Editor:
The code example does not illustrate the intended behavior. I am changing the example to be "ser[-1]" instead of "ser[:2]" and added a note that slicing with integers ignores the integer labels

Row 'c' is populated with 0.0, not NaN.

Note from the Author or Editor:
Confirmed. Will fix in the 3rd edition revision.

Looks like instead of method "match" there should be "get_indexer"

Note from the Author or Editor:
Fixing this to "get_indexer"

For the argument "names", combining with "header=None" is not needed. Using the parameter "names" implies this.

Note from the Author or Editor:
Right. Will fix

The 'iterator' option of pandas.read_csv returns a TextFileReader object, not a TextParser object, since 2012.

Note from the Author or Editor:
I am changing the test to correspond to changes in the latest version of pandas

As Out[38] shows, the 'iterator' option of pandas.read_csv returns a TextFileReader object, not a TextParser object, since 2012.

Note from the Author or Editor:
I am changing the text to correspond with changes in pandas

As Out[38] show, the 'iterator' option of pandas.read_csv returns a TextFileReader object, not a TextParser object, since 2012.

Note from the Author or Editor:
I am changing the text to correspond to the current version of pandas

In [64]: result
Out[64]:
{'name': 'Wes',
'pet': None,
'places_lived': ['United States', 'Spain', 'Germany'],
'siblings': [{'age': 30, 'name': 'Scott', 'pets': ['Zeus', 'Zuko']},
{'age': 38, 'name': 'Katie', 'pets': ['Sixes', 'Stache', 'Cisco']}]}

'pet': None,
Should print after the line:
'places_lived': ['United States', 'Spain', 'Germany'],

Note from the Author or Editor:
will fix

Code says the example file is in path:

path='examples/mta_perf/Performance_MNR.xml'

Actual path from git repository is:

path='datasets/mta_perf/Performance_MNR.xml'

Note from the Author or Editor:
Correct, thank you. This will need to be fixed in the source files

The command "store" at this point produces only the first two lines of the indicated output - the rest is not produced.

Note from the Author or Editor:
I think this was caused by a malformed build where the "mydata.h5" persisted between builds. I will see that it's fixed

In [35]: _ = df.fillna(0, inplace=True)

The assignment "_ =" is unnecessary.

Note from the Author or Editor:
will fix

In [85]: data = np.random.randn(20)

In [86]: pd.cut(data, 4, precision = 2)
.
.
.
The precision = 2 option limits the decimal precision to two digits.

-----------

However, one of the bins I get is (0.031, 0.27]

Note from the Author or Editor:
will investigate

The parameter "engine='python'" is needed in this command. Without this, a ParserWarning is produced due to the two character separator.

Book say: "match - Use re.match with the passed regular expression on each element, returning matched groups as list"

Should say: "... returning Series/array of boolean values"

And commands on pp 217 - 218 are not correct, because they return boolean values and there is no "access elements" at all.
Instead of:
In [174]: matches = data.str.match(pattern, flags=re.IGNORECASE)
In [175]: matches
Out[175]:
Dave True
Rob True
Steve True
Wes NaN
dtype: object

In [176]: matches.str.get(1)
Out[176]:Dave NaN
Rob NaN
Steve NaN
Wes NaN
dtype: float64

In [177]: matches.str[0]
Out[177]:
Dave NaN
Rob NaN
Steve NaN
Wes NaN
dtype: float64

it may be better to use:
In [174]: matches = data.str.extract(pattern, flags=re.IGNORECASE)
In [175]: matches
Out[175]:
0 1 2
Dave dave google com
Rob rob gmail com
Steve steve gmail com
Wes NaN NaN NaN

In [176]: matches[0]
Out[176]:
Dave dave
Rob rob
Steve steve
Wes NaN
Name: 0, dtype: object

In [177]: matches.iloc[:, 0]
Out[177]:
Dave dave
Rob rob
Steve steve
Wes NaN
Name: 0, dtype: object

Note from the Author or Editor:
This behavior changed in pandas. I'm correcting the code examples and the language in the text

The code

MultiIndex.from_arrays([['Ohio', 'Ohio', 'Colorado'], ['Green', 'Red', 'Green']],
names=['state', 'color'])

should be

pd.MultiIndex.from_arrays([['Ohio', 'Ohio', 'Colorado'], ['Green', 'Red', 'Green']],
names=['state', 'color'])

Note from the Author or Editor:
Adding the "pd."

Final entry in table is 'output' join.
It should be 'outer' join.

Note from the Author or Editor:
This needs to be changed from "output" to "outer"

The operation at the bottom of page 241:
In [112]: np.where(pd.isnull(a), b, a)
will take elements from a where available and from b where not available in a.

The analogous operation using combine_first should then probably be:
a.combine_first(b)
rather than:
b.combine_first(a)

Note from the Author or Editor:
will fix

In [132] And Out[132]
are the repetitions of:
In[131] And Out[131]

In [132] And Out[132] should be removed!

Note from the Author or Editor:
Confirmed. Will remove

In:

subplots_adjust(left=None, bottom=None, right=None, top=None, wspace=None, hspace=None)

None does not adjust. Use 0.

Note from the Author or Editor:
will fix

result = grouped['tip_pct', 'total_bill'].agg(functions) —-> needs an extra pair of []

Correct ——> result = grouped[['tip_pct', 'total_bill']].agg(functions)

Note from the Author or Editor:
Confirmed. Will fix in 3rd edition based on latest version of pandas

"for suit in ['H','S','C','D']: " should be "for suit in suits:". Otherwise, there is not point in defining "suits" earlier in the code block.

Note from the Author or Editor:
will fix

Need parameter "rcond=None" to suppress FutureWarning.

Note from the Author or Editor:
will fix

After the last sentence "Then, these can be concatenated together with concat:", it looks some python codes would be needed to make sense. These codes are found in https://github.com/wesm/pydata-book/blob/2nd-edition/ch14.ipynb , the below:

nutrients = []

for rec in db:
fnuts = pd.DataFrame(rec['nutrients'])
fnuts['id'] = rec['id']
nutrients.append(fnuts)

nutrients = pd.concat(nutrients, ignore_index=True)

Note from the Author or Editor:
Thanks -- I am restoring the code to the text (it was being accidentally suppressed in the output)

The paragraph states that "the output of outer will have a dimension that is the sum of the dimensions of the inputs".

Since the result of outer for (3, 4) and (5,) is (3, 4, 5), is it better to replace the word "sum" with "concatenation"?

Note from the Author or Editor:
Making suggested change

It would be better to make a zipped result more pretty for the last code example as follows:

In [188]: zip(last_name[sorter], first_name[sorter])
Out[188]: <zip at 0x7fa203eda1c8>

Note from the Author or Editor:
Adding "list(...)" to make the example prettier

"mmap" is a fairly large file on disk. It would be good to add a command to delete it when done here.

Note from the Author or Editor:
will fix

[more on same issue]

On my PC, I found that sum(0) runs faster on arr_c :

In [17]: %timeit arr_c.sum(0)
953 µs ± 9.66 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [18]: %timeit arr_f.sum(0)
1.6 ms ± 2.72 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)


I wonder if the output in [230] and [231] does not actually result from what was built in [225] and [226].

Note from the Author or Editor:
will review and make the difference more stark / consistent

The comment that deleting a variable does not free up memory appears to be incorrect. After using del I had a decrease in memory used on my mac as shown on activity monitor.

Note from the Author or Editor:
will fix

Found two syntax errors in Python3.

1)
for _ in xrange(niter):
needed to be replaced by like
for _ in range(niter):

2)
print 'Largest one we saw: %s' % np.max(some_results)
needed to be replaced by like
print('Largest one we saw: {0}'.format(np.max(some_results)))

Note from the Author or Editor:
Fixing this