Chapter 3
Sigmoid Neurons and Backpropagation
In this chapter, we describe the basic learning algorithm, which virtually all neural-network learning algorithms are variations of. This algorithm is based on a technique known as backpropagation (or just backprop) and was introduced in the context of neural networks in the mid-1980s. It was a significant step on the path to deep learning (DL). Our impression is that even to many DL practitioners, this algorithm can be a little bit of a mystery because much of it is hidden under the hood of modern DL frameworks. Still, it is crucial to know the basics of how the algorithm works.
At the highest level, the algorithm consists of three simple steps. First, present one or more training examples to the ...
Get Learning Deep Learning: Theory and Practice of Neural Networks, Computer Vision, NLP, and Transformers using TensorFlow now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
