Errata

Deep Learning from Scratch

Errata for Deep Learning from Scratch

Submit your own errata for this product.

The errata list is a list of errors and their corrections that were found after the product was released. If the error was corrected in a later version or reprint the date of the correction will be displayed in the column titled "Date Corrected".

The following errata were submitted by our customers and approved as valid errors by the author or editor.

Color key: Serious technical mistake Minor technical mistake Language or formatting error Typo Question Note Update

Version Location Description Submitted By Date submitted Date corrected
Page web
Figure 1.1

The function in figure 1.1 is not Relu but Leaky Relu.

Note from the Author or Editor:
correct, graph is mislabeled

 Jaap van der Does  Oct 16, 2019 
Page Chap 1
Figure 1.18

Shouldn't the symbol in the second blue box should be a \sigma and not \delta?

 Venkatesh-Prasad Ranganath  Nov 10, 2019 
Page Chapter 1
Figure 1.1

The figure says ReLU function, but instead plots Leaky ReLU

 Tamirlan Seidakhmetov  Jan 05, 2020 
Page Chaptet 1
"The Fun Part: The Backward Pass" -> "Code" section

Here "then increasing x11 by 0.001 should increase L by 0.01 × 0.2489", 0.01 should be changed to 0.001

 Tamirlan Seidakhmetov  Jan 07, 2020 
Printed
Page p. 84
2nd to last paragraph

default activation is said to be "Linear", but in the code snippet it is actually "Sigmoid". So in the code snippet on p.91, the linear_regression neural network would need an explicit assignment of the activation to Linear(), otherwise Sigmoid() would be used.

Note from the Author or Editor:
Reader is correct; this is fixed on the book GitHub page:
github.com/SethHWeidman/DLFS_code/blob/master/03_dlfs/Code.ipynb

 Anonymous  Oct 27, 2020 
ePub
Page https://learning.oreilly.com/library/view/deep-learning-from/9781492041405/ch01.html
John Cochrane, [Investments] Notes 2006

John Cochrane, [Investments] Notes 2006 => hyperlink is broken

Note from the Author or Editor:
Gokce is correct, unfortunately the link seems to have been taken down

 Gökçe Aydos  Sep 22, 2023 
ePub
Page Appendix - Matrix Chain Rule
"it isn’t too hard to see that the partial derivative of this with respect to x1"

> it isn’t too hard to see that the partial derivative of this with respect to `x_1`

`x_1` => `x_11`.

Note from the Author or Editor:
Seems like another ePub issue - I’m not seeing this error on O'Reilly Platform

 Gökçe Aydos  Sep 28, 2023 
Printed
Page 10
return statement in def chain_length_2() function

In chain_length_2() function, the return statement is f2(f1(x)) but x is undefined. The return statement should be f2(f1(a)) which a is the input for the function.

Note from the Author or Editor:
Anon is correct, this is a regrettable, sloppy error!

 Anonymous  Oct 08, 2019 
PDF
Page 10
Figure 1-7

The use of f1 f2 to indicate the composite f2(f1(x)) is confusing and non-standard. If the author wanted to pipe the functions sequentially to create the composite above then there is a standard way of doing this. Otherwise it should be simply noted.

As it currently stands the notation does not imply composition (and if so, incorrectly) but rather multiplication.

Solution:
There is a standard way of doing what the author wants. David Spivak at MIT uses this for an applied category theory course in mitpress.mit.edu/books/category-theory-sciences

Use a semicolon.

f1 ; f2 so this implies first apply f1 then apply f2 to the result.

Note from the Author or Editor:
This is a thoughtful suggestion from Bradford! My notation there wasn’t great, and I like his idea to make it better.

 Bradford Fournier-Eaton  Nov 01, 2021 
PDF
Page 11
the Math formula

1) Page 11 - the math formula.
From math standpoint this formula is incorrect.

The LHS is not the derivative of f2 w.r.t. x,
it is the derivative of the composite function "(f2 ∘ f1)" w.r.t. x.
I am sure that's what the author meant but that's not what's written.

So the author should probably denote this composite function as some letter e.g. h and then the LHS can be written as dh/du. Then the formula would be OK.

2) Also figure 1.7 on the previous page is incorrect. This composite function is not f1 . f2, it is f2 . f1 (this is read as f2 after f1), or as written usually in math terms (f2 ∘ f1).

3) Same error on figure 1.8 (page 12). Same error f1 . f2 on the first line of page 11.

4) Also, the terms nested functions and composite functions are used interchangeably which is very very confusing. In programming the term nested functions has a different meaning usually, it is when a function is defined inside another function's body. This is not what the word is about here.

So this whole sequence of pages starting at page 9 needs serious editing, it's full of confusions.











Note from the Author or Editor:
I agree with #2 above - one component of a formula written as “f2(x)” should be “f2(f1(x))”. The rest of the comments are really stylistic opinions.

 Peter Petrov  Mar 26, 2021 
PDF
Page 13
In the function : chain_deriv_2

# df1/dx
f1_of_x = f1(input_range)

In the above line f1_of_x calculates applied function f1 over the input data , the comment suggest its calculating derivative which is not true.

Note from the Author or Editor:
Pradeep is correct, the code comment is misleading

 Pradeep Kumar  Oct 10, 2020 
PDF
Page 13
In the function : chain_deriv_2

There is no where it is mentioned what is plot_chain does. No codes are given in that chapter for reference neither its clear what does it do. This function is being used everywhere in the first chapter

In the type annotations of ndarray it should be either np.ndarray or numpy.ndarray

Note from the Author or Editor:
Helpful comment from Pradeep, I could have included code for the `plot_chain` function

 Pradeep Kumar  Oct 10, 2020 
Printed
Page 25
Last paragraph

The text reads "...the gradient of X with respect to X." but it should read "...the gradient of N with respect to X." A gradient is a property of a function, not a vector.

 Jason Gastelum  Dec 25, 2020 
Printed
Page 28
Chapter 1

"we compute quantities on the forward pass (here, just N)"
contradicts what was defined as "forward pass" on p. 16 in the chain_deriv_3() method.
According to that definition, the foward pass comprises
N = np.dot(,X, W)
as well as
S = sigma(N)

Note from the Author or Editor:
I can see why Jason is confused - there are two quantities specified in the code, N and S, that are saved on the forward pass to be used in the backwardpass. I could clean up this language.

 Anonymous  Apr 29, 2020 
Printed
Page 64
Tabel 2-1 Derivative table for neural network

the partial derivative dLdP = -(forward_info[y] - forward_info[p]) should be -2 * (forward_info[y] - forward_info[p]), just like the explanation on page 51.

Anonymous  Oct 25, 2019 
Printed
Page 65
2nd


In the source code, there is a bug while calculating the gradients with respect to W2:

github.com/SethHWeidman/DLFS_code/blob/master/02_fundamentals/Code.ipynb

Incorrect: dLdW2 = np.dot(dM2dW2, dLdP)
Correct: dLdW2 = np.dot(dM2dW2, dLdM2)

Note from the Author or Editor:
This is technically correct (even though it results in the same computation) - great catch by Eugen.

 Eugen Grosu  Jan 03, 2021 
Printed, PDF
Page 88
section heading

Heading is the same as the chapter title *and* the book title.

"In DLfS refer to the section DLfS in the chapter DLfS" might be a little confusing.

Note from the Author or Editor:
Ha, fair point!

 Anonymous  Feb 29, 2020 
Printed
Page 91
NeuralNetwork class invocations in the code

The NeuralNetwork class, when used on page 91, is given a learning_rate parameter --- there's no learning_rate in the __init__ function for that class, and no methods in the class use the learning_rate. This is not surprising, as the learning-rate is something the Optimizer class (introduced on the following pages) cares about.

Note from the Author or Editor:
Great catch - I should have cleaned this up

 David Mankins  Sep 13, 2023 
PDF
Page 94
__init__ method of class Trainer

The __init__ method is missing self.optim = optim before the setattr line.

Note from the Author or Editor:
Rodrigo is correct; this is corrected here: github.com/SethHWeidman/DLFS_code/blob/bfeb64746d88f7f7add4499b35b324765ff34268/lincoln/lincoln/train.py

 Rodrigo Stevaux  Oct 07, 2020 
ePub
Page 99
1st paragraph

In the Lincoln library, required to run the cpde for chapter 4, 'lincoln.utils.np_utils' does not contain the function 'exp_ratios'.

Note from the Author or Editor:
Great call, this function was taken out

 Steven Kaminsky  Jan 14, 2020