Deep Learning at Scale

Chapter 2. Deep Learning

Chapter 1 covered the impact of scale on the history and evolution of deep learning. This chapter dives into the internal mechanics of deep learning and includes two practical Hands-On Exercises to warm up your deep learning skills. You will investigate the role of data in deep learning and explore several concepts involved in model development, using a minimalistic Python implementation of deep learning. You’ll get the opportunity to apply these learnings by building a PyTorch-based model and exploring some important and interesting auxiliary utilities that are needed for all scaling efforts. This chapter is very practice-oriented and does not dive deeply into the theoretical foundations of deep learning. Please refer to Deep Learning by Aaron Courville, Ian Goodfellow, and Yoshua Bengio if you are seeking theoretical foundational knowledge.¹

The Role of Data in Deep Learning

All models are wrong, but some are useful.

—George Box

Deep learning is a data programming technique that uses algorithmic approaches to obtain results given the input variable(s). Deep learning relies heavily on mathematics and statistics to obtain a model $M$ such that result y can be obtained by applying $y\leftarrow M\left(x\right)$ to the given input x. The methodology for obtaining the model $M$ in deep learning differs from the traditional stochastic data modeling techniques, such as linear or logistic regression. Traditional data modeling techniques heavily emphasize the goodness of ...