Deep Learning with TensorFlow 2 and Keras - Second Edition

Deep Learning with TensorFlow 2 and Keras - Second Edition

by Antonio Gulli, Amita Kapoor, Sujit Pal
Released December 2019
Publisher(s): Packt Publishing
ISBN: 9781838823412

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Build machine and deep learning systems with the newly released TensorFlow 2 and Keras for the lab, production, and mobile devices

Key Features

  • Introduces and then uses TensorFlow 2 and Keras right from the start
  • Teaches key machine and deep learning techniques
  • Understand the fundamentals of deep learning and machine learning through clear explanations and extensive code samples

Book Description

Deep Learning with TensorFlow 2 and Keras, Second Edition teaches neural networks and deep learning techniques alongside TensorFlow (TF) and Keras. You'll learn how to write deep learning applications in the most powerful, popular, and scalable machine learning stack available.

TensorFlow is the machine learning library of choice for professional applications, while Keras offers a simple and powerful Python API for accessing TensorFlow. TensorFlow 2 provides full Keras integration, making advanced machine learning easier and more convenient than ever before.

This book also introduces neural networks with TensorFlow, runs through the main applications (regression, ConvNets (CNNs), GANs, RNNs, NLP), covers two working example apps, and then dives into TF in production, TF mobile, and using TensorFlow with AutoML.

What you will learn

  • Build machine learning and deep learning systems with TensorFlow 2 and the Keras API
  • Use Regression analysis, the most popular approach to machine learning
  • Understand ConvNets (convolutional neural networks) and how they are essential for deep learning systems such as image classifiers
  • Use GANs (generative adversarial networks) to create new data that fits with existing patterns
  • Discover RNNs (recurrent neural networks) that can process sequences of input intelligently, using one part of a sequence to correctly interpret another
  • Apply deep learning to natural human language and interpret natural language texts to produce an appropriate response
  • Train your models on the cloud and put TF to work in real environments
  • Explore how Google tools can automate simple ML workflows without the need for complex modeling

Who this book is for

This book is for Python developers and data scientists who want to build machine learning and deep learning systems with TensorFlow. Whether or not you have done machine learning before, this book gives you the theory and practice required to use Keras, TensorFlow 2, and AutoML to build machine learning systems.

Table of contents

  1. Preface
    1. Mission
    2. Machine learning, artificial intelligence, and the deep learning Cambrian explosion
    3. Who this book is for
    4. What this book covers
    5. What you need for this book
      1. Downloading the example code
      2. Download the color images
      3. Conventions
    6. Get in touch
      1. Reviews
    7. References
  2. Neural Network Foundations with TensorFlow 2.0
    1. What is TensorFlow (TF)?
    2. What is Keras?
    3. What are the most important changes in TensorFlow 2.0?
    4. Introduction to neural networks
    5. Perceptron
      1. A first example of TensorFlow 2.0 code
    6. Multi-layer perceptron – our first example of a network
      1. Problems in training the perceptron and their solutions
      2. Activation function – sigmoid
      3. Activation function – tanh
      4. Activation function – ReLU
      5. Two additional activation functions – ELU and LeakyReLU
      6. Activation functions
      7. In short – what are neural networks after all?
    7. A real example – recognizing handwritten digits
      1. One-hot encoding (OHE)
      2. Defining a simple neural network in TensorFlow 2.0
      3. Running a simple TensorFlow 2.0 net and establishing a baseline
      4. Improving the simple net in TensorFlow 2.0 with hidden layers
      5. Further improving the simple net in TensorFlow with Dropout
      6. Testing different optimizers in TensorFlow 2.0
      7. Increasing the number of epochs
      8. Controlling the optimizer learning rate
      9. Increasing the number of internal hidden neurons
      10. Increasing the size of batch computation
      11. Summarizing experiments run for recognizing handwritten charts
    8. Regularization
      1. Adopting regularization to avoid overfitting
      2. Understanding BatchNormalization
    9. Playing with Google Colab – CPUs, GPUs, and TPUs
    10. Sentiment analysis
    11. Hyperparameter tuning and AutoML
    12. Predicting output
    13. A practical overview of backpropagation
    14. What have we learned so far?
    15. Towards a deep learning approach
    16. References
  3. TensorFlow 1.x and 2.x
    1. Understanding TensorFlow 1.x
      1. TensorFlow 1.x computational graph program structure
        1. Computational graphs
      2. Working with constants, variables, and placeholders
      3. Examples of operations
        1. Constants
        2. Sequences
        3. Random tensors
        4. Variables
      4. An example of TensorFlow 1.x in TensorFlow 2.x
    2. Understanding TensorFlow 2.x
      1. Eager execution
      2. AutoGraph
      3. Keras APIs – three programming models
        1. Sequential API
        2. Functional API
        3. Model subclassing
      4. Callbacks
      5. Saving a model and weights
      6. Training from tf.data.datasets
      7. tf.keras or Estimators?
      8. Ragged tensors
      9. Custom training
      10. Distributed training in TensorFlow 2.x
        1. Multiple GPUs
        2. MultiWorkerMirroredStrategy
        3. TPUStrategy
        4. ParameterServerStrategy
      11. Changes in namespaces
      12. Converting from 1.x to 2.x
      13. Using TensorFlow 2.x effectively
    3. The TensorFlow 2.x ecosystem
      1. Language bindings
    4. Keras or tf.keras?
    5. Summary
  4. Regression
    1. What is regression?
    2. Prediction using linear regression
      1. Simple linear regression
      2. Multiple linear regression
      3. Multivariate linear regression
    3. TensorFlow Estimators
      1. Feature columns
      2. Input functions
      3. MNIST using TensorFlow Estimator API
    4. Predicting house price using linear regression
    5. Classification tasks and decision boundaries
      1. Logistic regression
      2. Logistic regression on the MNIST dataset
    6. Summary
    7. References
  5. Convolutional Neural Networks
    1. Deep Convolutional Neural Network (DCNN)
      1. Local receptive fields
      2. Shared weights and bias
      3. A mathematical example
      4. ConvNets in TensorFlow 2.x
      5. Pooling layers
        1. Max pooling
        2. Average pooling
        3. ConvNets summary
    2. An example of DCNN ‒ LeNet
      1. LeNet code in TensorFlow 2.0
      2. Understanding the power of deep learning
    3. Recognizing CIFAR-10 images with deep learning
      1. Improving the CIFAR-10 performance with a deeper network
      2. Improving the CIFAR-10 performance with data augmentation
      3. Predicting with CIFAR-10
    4. Very deep convolutional networks for large-scale image recognition
      1. Recognizing cats with a VGG16 Net
      2. Utilizing tf.keras built-in VGG16 Net module
      3. Recycling prebuilt deep learning models for extracting features
    5. Summary
    6. References
  6. Advanced Convolutional Neural Networks
    1. Computer vision
      1. Composing CNNs for complex tasks
        1. Classification and localization
        2. Semantic segmentation
        3. Object detection
        4. Instance segmentation
      2. Classifying Fashion-MNIST with a tf.keras - estimator model
      3. Run Fashion-MNIST the tf.keras - estimator model on GPUs
      4. Deep Inception-v3 Net used for transfer learning
      5. Transfer learning for classifying horses and humans
      6. Application Zoos with tf.keras and TensorFlow Hub
        1. Keras applications
        2. TensorFlow Hub
      7. Other CNN architectures
        1. AlexNet
        2. Residual networks
        3. HighwayNets and DenseNets
        4. Xception
      8. Answering questions about images (VQA)
      9. Style transfer
        1. Content distance
        2. Style distance
      10. Creating a DeepDream network
      11. Inspecting what a network has learned
    2. Video
      1. Classifying videos with pretrained nets in six different ways
    3. Textual documents
      1. Using a CNN for sentiment analysis
    4. Audio and music
      1. Dilated ConvNets, WaveNet, and NSynth
    5. A summary of convolution operations
      1. Basic convolutional neural networks (CNN or ConvNet)
      2. Dilated convolution
        1. Transposed convolution
      3. Separable convolution
      4. Depthwise convolution
      5. Depthwise separable convolution
    6. Capsule networks
      1. So what is the problem with CNNs?
      2. So what is new with Capsule networks?
    7. Summary
    8. References
  7. Generative Adversarial Networks
    1. What is a GAN?
      1. MNIST using GAN in TensorFlow
    2. Deep convolutional GAN (DCGAN)
      1. DCGAN for MNIST digits
    3. Some interesting GAN architectures
      1. SRGAN
      2. CycleGAN
      3. InfoGAN
    4. Cool applications of GANs
    5. CycleGAN in TensorFlow 2.0
    6. Summary
    7. References
  8. Word Embeddings
    1. Word embedding ‒ origins and fundamentals
    2. Distributed representations
    3. Static embeddings
      1. Word2Vec
      2. GloVe
    4. Creating your own embedding using gensim
    5. Exploring the embedding space with gensim
    6. Using word embeddings for spam detection
      1. Getting the data
      2. Making the data ready for use
      3. Building the embedding matrix
      4. Define the spam classifier
      5. Train and evaluate the model
      6. Running the spam detector
    7. Neural embeddings – not just for words
      1. Item2Vec
      2. node2vec
    8. Character and subword embeddings
    9. Dynamic embeddings
    10. Sentence and paragraph embeddings
    11. Language model-based embeddings
      1. Using BERT as a feature extractor
      2. Fine-tuning BERT
      3. Classifying with BERT ‒ command line
      4. Using BERT as part of your own network
    12. Summary
    13. References
  9. Recurrent Neural Networks
    1. The basic RNN cell
      1. Backpropagation through time (BPTT)
      2. Vanishing and exploding gradients
    2. RNN cell variants
      1. Long short-term memory (LSTM)
      2. Gated recurrent unit (GRU)
      3. Peephole LSTM
    3. RNN variants
      1. Bidirectional RNNs
      2. Stateful RNNs
    4. RNN topologies
      1. Example ‒ One-to-Many – learning to generate text
      2. Example ‒ Many-to-One – Sentiment Analysis
      3. Example ‒ Many-to-Many – POS tagging
    5. Encoder-Decoder architecture – seq2seq
      1. Example ‒ seq2seq without attention for machine translation
    6. Attention mechanism
      1. Example ‒ seq2seq with attention for machine translation
    7. Transformer architecture
    8. Summary
    9. References
  10. Autoencoders
    1. Introduction to autoencoders
    2. Vanilla autoencoders
      1. TensorFlow Keras layers ‒ defining custom layers
      2. Reconstructing handwritten digits using an autoencoder
    3. Sparse autoencoder
    4. Denoising autoencoders
      1. Clearing images using a Denoising autoencoder
    5. Stacked autoencoder
      1. Convolutional autoencoder for removing noise from images
      2. Keras autoencoder example ‒ sentence vectors
    6. Summary
    7. References
  11. Unsupervised Learning
    1. Principal component analysis
      1. PCA on the MNIST dataset
      2. TensorFlow Embedding API
      3. K-means clustering
      4. K-means in TensorFlow 2.0
      5. Variations in k-means
    2. Self-organizing maps
      1. Colour mapping using SOM
    3. Restricted Boltzmann machines
      1. Reconstructing images using RBM
      2. Deep belief networks
    4. Variational Autoencoders
    5. Summary
    6. References
  12. Reinforcement Learning
    1. Introduction
      1. RL lingo
      2. Deep reinforcement learning algorithms
      3. Reinforcement success in recent years
    2. Introduction to OpenAI Gym
      1. Random agent playing Breakout
    3. Deep Q-Networks
      1. DQN for CartPole
      2. DQN to play a game of Atari
      3. DQN variants
        1. Double DQN
        2. Dueling DQN
        3. Rainbow
    4. Deep deterministic policy gradient
    5. Summary
    6. References
  13. TensorFlow and Cloud
    1. Deep learning on cloud
      1. Microsoft Azure
      2. Amazon Web Services (AWS)
      3. Google Cloud Platform (GCP)
      4. IBM Cloud
    2. Virtual machines on cloud
      1. EC2 on Amazon
      2. Compute Instance on GCP
      3. Virtual machine on Microsoft Azure
    3. Jupyter Notebooks on cloud
      1. SageMaker
      2. Google Colaboratory
      3. Microsoft Azure Notebooks
    4. TensorFlow Extended for production
      1. TFX Pipelines
      2. TFX pipeline components
      3. TFX libraries
    5. TensorFlow Enterprise
    6. Summary
    7. References
  14. TensorFlow for Mobile and IoT and TensorFlow.js
    1. TensorFlow Mobile
    2. TensorFlow Lite
      1. Quantization
      2. FlatBuffers
      3. Mobile converter
      4. Mobile optimized interpreter
      5. Supported platforms
      6. Architecture
      7. Using TensorFlow Lite
      8. A generic example of application
      9. Using GPUs and accelerators
      10. An example of application
    3. Pretrained models in TensorFlow Lite
      1. Image classification
      2. Object detection
      3. Pose estimation
      4. Smart reply
      5. Segmentation
      6. Style transfer
      7. Text classification
      8. Question and answering
      9. A note about using mobile GPUs
    4. An overview of federated learning at the edge
      1. TensorFlow FL APIs
    5. TensorFlow.js
      1. Vanilla TensorFlow.js
      2. Converting models
      3. Pretrained models
      4. Node.js
    6. Summary
    7. References
  15. An introduction to AutoML
    1. What is AutoML?
    2. Achieving AutoML
    3. Automatic data preparation
    4. Automatic feature engineering
    5. Automatic model generation
    6. AutoKeras
    7. Google Cloud AutoML
      1. Using Cloud AutoML ‒ Tables solution
      2. Using Cloud AutoML ‒ Vision solution
      3. Using Cloud AutoML ‒ Text Classification solution
      4. Using Cloud AutoML ‒ Translation solution
      5. Using Cloud AutoML ‒ Video Intelligence Classification solution
      6. Cost
    8. Bringing Google AutoML to Kaggle
    9. Summary
    10. References
  16. The Math Behind Deep Learning
    1. History
    2. Some mathematical tools
      1. Derivatives and gradients everywhere
      2. Gradient descent
      3. Chain rule
      4. A few differentiation rules
      5. Matrix operations
    3. Activation functions
      1. Derivative of the sigmoid
      2. Derivative of tanh
      3. Derivative of ReLU
    4. Backpropagation
      1. Forward step
      2. Backstep
        1. Case 1 – From hidden layer to output layer
        2. Case 2 ‒ From hidden layer to hidden layer
      3. Limit of backpropagation
      4. Cross entropy and its derivative
      5. Batch gradient descent, stochastic gradient descent, and mini-batch
        1. Batch Gradient Descent (BGD)
        2. Stochastic Gradient Descent (SGD)
        3. Mini-Batch Gradient Descent (MBGD)
    5. Thinking about backpropagation and convnets
    6. Thinking about backpropagation and RNNs
    7. A note on TensorFlow and automatic differentiation
    8. Summary
    9. References
  17. Tensor Processing Unit
    1. C/G/T processing units
      1. CPUs and GPUs
      2. TPUs
    2. Three generations of TPUs and Edge TPU
      1. First-generation TPU
      2. Second-generation TPU
      3. Third-generation TPU
      4. Edge TPU
    3. TPU performance
    4. How to use TPUs with Colab
      1. Checking whether TPUs are available
      2. Loading data with tf.data
      3. Building a model and loading it into the TPU
    5. Using pretrained TPU models
    6. Using TensorFlow 2.1 and nightly build
    7. Summary
    8. References
  18. Other Books You May Enjoy
  19. Index

Product information

  • Title: Deep Learning with TensorFlow 2 and Keras - Second Edition
  • Author(s): Antonio Gulli, Amita Kapoor, Sujit Pal
  • Release date: December 2019
  • Publisher(s): Packt Publishing
  • ISBN: 9781838823412