Deep Learning for Beginners

Deep Learning for Beginners

by Dr. Pablo Rivas
Released September 2020
Publisher(s): Packt Publishing
ISBN: 9781838640859

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Implement supervised, unsupervised, and generative deep learning (DL) models using Keras and Dopamine with TensorFlow

Key Features

  • Understand the fundamental machine learning concepts useful in deep learning
  • Learn the underlying mathematical concepts as you implement deep learning models from scratch
  • Explore easy-to-understand examples and use cases that will help you build a solid foundation in DL

Book Description

With information on the web exponentially increasing, it has become more difficult than ever to navigate through everything to find reliable content that will help you get started with deep learning. This book is designed to help you if you're a beginner looking to work on deep learning and build deep learning models from scratch, and you already have the basic mathematical and programming knowledge required to get started.

The book begins with a basic overview of machine learning, guiding you through setting up popular Python frameworks. You will also understand how to prepare data by cleaning and preprocessing it for deep learning, and gradually go on to explore neural networks. A dedicated section will give you insights into the working of neural networks by helping you get hands-on with training single and multiple layers of neurons. Later, you will cover popular neural network architectures such as CNNs, RNNs, AEs, VAEs, and GANs with the help of simple examples, and learn how to build models from scratch. At the end of each chapter, you will find a question and answer section to help you test what you've learned through the course of the book.

By the end of this book, you'll be well-versed with deep learning concepts and have the knowledge you need to use specific algorithms with various tools for different tasks.

What you will learn

  • Implement recurrent neural networks (RNNs) and long short-term memory (LSTM) for image classification and natural language processing tasks
  • Explore the role of convolutional neural networks (CNNs) in computer vision and signal processing
  • Discover the ethical implications of deep learning modeling
  • Understand the mathematical terminology associated with deep learning
  • Code a generative adversarial network (GAN) and a variational autoencoder (VAE) to generate images from a learned latent space
  • Implement visualization techniques to compare AEs and VAEs

Who this book is for

This book is for aspiring data scientists and deep learning engineers who want to get started with the fundamentals of deep learning and neural networks. Although no prior knowledge of deep learning or machine learning is required, familiarity with linear algebra and Python programming is necessary to get started.

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. Deep Learning for Beginners
  3. About Packt
    1. Why subscribe?
  4. Foreword
  5. Contributors
    1. About the author
    2. About the reviewers
    3. Packt is searching for authors like you
  6. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Download the color images
      3. Conventions used
    4. Get in touch
      1. Reviews
  7. Section 1: Getting Up to Speed
  8. Introduction to Machine Learning
    1. Diving into the ML ecosystem
    2. Training ML algorithms from data
    3. Introducing deep learning
      1. The model of a neuron
      2. The perceptron learning algorithm
      3. Shallow networks
        1. The input-to-hidden layer
        2. The hidden-to-hidden layer
        3. The hidden-to-output layer
      4. Deep networks
    4. Why is deep learning important today?
    5. Summary 
    6. Questions and answers
    7. References
  9. Setup and Introduction to Deep Learning Frameworks
    1. Introduction to Colaboratory
    2. Introduction and setup of TensorFlow
      1. Setup
        1. TensorFlow with GPU support
      2. Principles behind TensorFlow
    3. Introduction and setup of Keras
      1. Setup
      2. Principles behind Keras
    4. Introduction to PyTorch
    5. Introduction to Dopamine
    6. Other deep learning libraries
      1. Caffe
      2. Theano
      3. Honorable mentions
    7. Summary 
    8. Questions and answers
    9. References
  10. Preparing Data
    1. Binary data and binary classification
      1. Binary targets on the Cleveland Heart Disease dataset
      2. Binarizing the MNIST dataset
        1. Binarizing the images
        2. Binarizing the targets
    2. Categorical data and multiple classes
      1. Converting string labels to numbers
      2. Converting categories to one-hot encoding
    3. Real-valued data and univariate regression
      1. Scaling to a specific range of values
      2. Standardizing to zero mean and unit variance
    4. Altering the distribution of data
    5. Data augmentation
      1. Rescaling
      2. Adding noise
      3. Rotating
      4. Other augmentation techniques
    6. Data dimensionality reduction
      1. Supervised algorithms
        1. Linear discriminant analysis
      2. Unsupervised techniques
        1. Kernel PCA
        2. Large datasets
          1. Sparse PCA
          2. Dictionary Learning
      3. Regarding the number of dimensions
    7. Ethical implications of manipulating data
    8. Summary 
    9. Questions and answers
    10. References
  11. Learning from Data
    1. Learning for a purpose
      1. Classification
        1. Binary classification
        2. Multi-class classification
      2. Regression
    2. Measuring success and error
      1. Binary classification
      2. Multiple classes
      3. Regression
    3. Identifying overfitting and generalization
      1. If we have test data
      2. No test data? No problem – cross-validate
    4. The art behind learning
    5. Ethical implications of training deep learning algorithms
      1. Reporting using the appropriate performance measures
      2. Being careful with outliers and verifying them
      3. Weight classes with undersampled groups
    6. Summary 
    7. Questions and answers
    8. References
  12. Training a Single Neuron
    1. The perceptron model
      1. The visual concept
      2. Tensor operations
    2. The perceptron learning algorithm
      1. PLA in Python
    3. A perceptron over non-linearly separable data
      1. Convergence on linearly separable data
      2. Convergence on non-linearly separable data
    4. Summary 
    5. Questions and answers
    6. References
  13. Training Multiple Layers of Neurons
    1. The MLP model
    2. Minimizing the error
      1. Step 1 – Initialization
      2. Step 2 – The forward pass
      3. Step 3 – Calculating loss
      4. Step 4 – The backward pass
    3. Finding the best hyperparameters
    4. Summary 
    5. Questions and answers
    6. References
  14. Section 2: Unsupervised Deep Learning
  15. Autoencoders
    1. Introduction to unsupervised learning
    2. Encoding and decoding layers
      1. Encoding layers
      2. Decoding layers
      3. Loss function
      4. Learning and testing
    3. Applications in dimensionality reduction and visualization
      1. MNIST data preparation
      2. Autoencoders for MNIST 
      3. Training and visualization
    4. Ethical implications of unsupervised learning
    5. Summary 
    6. Questions and answers
    7. References
  16. Deep Autoencoders
    1. Introducing deep belief networks
    2. Making deep autoencoders
      1. Batch normalization
      2. Dropout
    3. Exploring latent spaces with deep autoencoders
      1. CIFAR-10
      2. MNIST 
    4. Summary 
    5. Questions and answers
    6. References
  17. Variational Autoencoders
    1. Introducing deep generative models
    2. Examining the VAE model
      1. The heart disease dataset revisited
      2. The re-parametrization trick and sampling 
      3. Learning the posterior's parameters in the encoder
      4. Modeling the decoder
      5. Minimizing the loss
      6. Training a VAE
      7. Generating data from the VAE
    3. Comparing a deep and shallow VAE on MNIST
      1. Shallow VAE
      2. Deep VAE
        1. Encoder
        2. Decoder
      3. Denoising VAEs
    4. Thinking about the ethical implications of generative models
    5. Summary 
    6. Questions and answers
    7. References
  18. Restricted Boltzmann Machines
    1. Introduction to RBMs
      1. BMs
      2. RBMs
      3. Bernoulli RBMs
    2. Learning data representations with RBMs
    3. Comparing RBMs and AEs
    4. Summary
    5. Questions and answers
    6. References
  19. Section 3: Supervised Deep Learning
  20. Deep and Wide Neural Networks
    1. Wide neural networks
      1. Deep learning revisited
      2. Wide layers
        1. Summaries
        2. Names
      3. The CIFAR-10 dataset
      4. New training tools
        1. Saving or loading models
        2. Reducing the learning rate on the fly
        3. Stopping the learning process early
      5. Results
    2. Dense deep neural networks
      1. Building and training the model
      2. Results
    3. Sparse deep neural networks
      1. Building a sparse network and training it
      2. Results
    4. Hyperparameter optimization
      1. Libraries and parameters
      2. Implementation and results
    5. Summary 
    6. Questions and answers
    7. References
  21. Convolutional Neural Networks
    1. Introduction to convolutional neural networks
    2. Convolution in n-dimensions
      1. 1-dimension
      2. 2-dimensions
      3. n-dimensions
    3. Convolutional layers
      1. Conv2D
      2. The layer+activation combo
    4. Pooling strategies
    5. Convolutional neural network for CIFAR-10
      1. Implementation
        1. Loading data
        2. Compiling the model
        3. Training the CNN
      2. Results
      3. Visualization of filters
    6. Summary
    7. Questions and answers
    8. References
  22. Recurrent Neural Networks
    1. Introduction to recurrent neural networks
      1. Simple RNNs
      2. Embedding layers
      3. Word embedding and RNNs on IMDb 
    2. Long short-term memory models
    3. Sequence-to-vector models
      1. Unsupervised model
      2. Results
    4. Vector-to-sequence models
      1. Bi-directional LSTM
      2. Implementation and results
    5. Sequence-to-sequence models
    6. Ethical implications
    7. Summary
    8. Questions and answers
    9. References
  23. Generative Adversarial Networks
    1. Introducing adversarial learning
      1. Learning using an adversary
      2. GANs
    2. Training a GAN
      1. An MLP model
      2. A convolutional model
    3. Comparing GANs and VAEs
    4. Thinking about the ethical implications of GANs
    5. Summary
    6. Questions and answers
    7. References
  24. Final Remarks on the Future of Deep Learning
    1. Looking for advanced topics in deep learning
      1. Deep reinforcement learning
      2. Self-supervised learning
      3. System 2 algorithms
    2. Learning with more resources from Packt
      1. Reinforcement learning
      2. Self-supervised learning
    3. Summary
    4. References
  25. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Deep Learning for Beginners
  • Author(s): Dr. Pablo Rivas
  • Release date: September 2020
  • Publisher(s): Packt Publishing
  • ISBN: 9781838640859