Hands-On Convolutional Neural Networks with TensorFlow

Hands-On Convolutional Neural Networks with TensorFlow

by Iffat Zafar, Giounona Tzanidou, Richard Burton, Nimesh Patel, Leonardo Araujo
Released August 2018
Publisher(s): Packt Publishing
ISBN: 9781789130331

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Learn how to apply TensorFlow to a wide range of deep learning and Machine Learning problems with this practical guide on training CNNs for image classification, image recognition, object detection and many computer vision challenges.

Key Features

  • Learn the fundamentals of Convolutional Neural Networks
  • Harness Python and Tensorflow to train CNNs
  • Build scalable deep learning models that can process millions of items

Book Description

Convolutional Neural Networks (CNN) are one of the most popular architectures used in computer vision apps. This book is an introduction to CNNs through solving real-world problems in deep learning while teaching you their implementation in popular Python library - TensorFlow. By the end of the book, you will be training CNNs in no time!

We start with an overview of popular machine learning and deep learning models, and then get you set up with a TensorFlow development environment. This environment is the basis for implementing and training deep learning models in later chapters. Then, you will use Convolutional Neural Networks to work on problems such as image classification, object detection, and semantic segmentation.

After that, you will use transfer learning to see how these models can solve other deep learning problems. You will also get a taste of implementing generative models such as autoencoders and generative adversarial networks.

Later on, you will see useful tips on machine learning best practices and troubleshooting. Finally, you will learn how to apply your models on large datasets of millions of images.

What you will learn

  • Train machine learning models with TensorFlow
  • Create systems that can evolve and scale during their life cycle
  • Use CNNs in image recognition and classification
  • Use TensorFlow for building deep learning models
  • Train popular deep learning models
  • Fine-tune a neural network to improve the quality of results with transfer learning
  • Build TensorFlow models that can scale to large datasets and systems

Who this book is for

This book is for Software Engineers, Data Scientists, or Machine Learning practitioners who want to use CNNs for solving real-world problems. Knowledge of basic machine learning concepts, linear algebra and Python will help.

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. Hands-On Convolutional Neural Networks with TensorFlow
  3. Packt Upsell
    1. Why subscribe?
    2. PacktPub.com
  4. Contributors
    1. About the authors
    2. Packt is searching for authors like you
  5. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Conventions used
    4. Get in touch
      1. Reviews
  6. Setup and Introduction to TensorFlow
    1. The TensorFlow way of thinking
    2. Setting up and installing TensorFlow
      1. Conda environments
      2. Checking whether your installation works
    3. TensorFlow API levels
      1. Eager execution
    4. Building your first TensorFlow model
      1. One-hot vectors
      2. Splitting into training and test sets
    5. Creating TensorFlow graphs
      1. Variables
      2. Operations
    6. Feeding data with placeholders
    7. Initializing variables
    8. Training our model
      1. Loss functions
      2. Optimization
    9. Evaluating a trained model
    10. The session
    11. Summary
  7. Deep Learning and Convolutional Neural Networks
    1. AI and ML
      1. Types of ML
      2. Old versus new ML
    2. Artificial neural networks
      1. Activation functions
      2. The XOR problem
      3. Training neural networks
      4. Backpropagation and the chain rule
      5. Batches
      6. Loss functions
      7. The optimizer and its hyperparameters
        1. Underfitting versus overfitting
      8. Feature scaling
      9. Fully connected layers
      10. A TensorFlow example for the XOR problem
    3. Convolutional neural networks
      1. Convolution
      2. Input padding
      3. Calculating the number of parameters (weights)
      4.  Calculating the number of operations
      5. Converting convolution layers into fully connected layers
      6. The pooling layer
      7. 1x1 Convolution
      8. Calculating the receptive field
    4. Building a CNN model in TensorFlow
      1. TensorBoard
      2. Other types of convolutions
    5. Summary
  8. Image Classification in TensorFlow
    1. CNN model architecture
      1. Cross-entropy loss (log loss)
      2. Multi-class cross entropy loss
      3. The train/test dataset split
    2. Datasets
      1. ImageNet
      2. CIFAR
      3. Loading CIFAR
    3. Image classification with TensorFlow
      1. Building the CNN graph
      2. Learning rate scheduling
      3. Introduction to the tf.data API
      4. The main training loop
    4. Model Initialization
      1. Do not initialize all weights with zeros
      2. Initializing with a mean zero distribution
        1. Xavier-Bengio and the Initializer
    5. Improving generalization by regularizing
      1. L2 and L1 regularization
      2. Dropout
      3. The batch norm layer
    6. Summary
  9. Object Detection and Segmentation
    1. Image classification with localization
      1. Localization as regression
      2. TensorFlow implementation
      3. Other applications of localization
    2. Object detection as classification – Sliding window
      1. Using heuristics to guide us (R-CNN)
        1. Problems
      2. Fast R-CNN
      3. Faster R-CNN
        1. Region Proposal Network
        2. RoI Pooling layer
      4. Conversion from traditional CNN to Fully Convnets
    3. Single Shot Detectors – You Only Look Once
      1. Creating training set for Yolo object detection
      2. Evaluating detection (Intersection Over Union)
      3. Filtering output
      4. Anchor Box
      5. Testing/Predicting in Yolo
      6. Detector Loss function (YOLO loss)
        1. Loss Part 1
        2. Loss Part 2
        3. Loss Part 3
    4. Semantic segmentation
      1. Max Unpooling
      2. Deconvolution layer (Transposed convolution)
      3. The loss function
      4. Labels
      5. Improving results
    5. Instance segmentation
      1. Mask R-CNN
    6. Summary
  10. VGG, Inception Modules, Residuals, and MobileNets
    1. Substituting big convolutions
      1. Substituting the 3x3 convolution
    2. VGGNet
      1. Architecture
      2. Parameters and memory calculation
      3. Code
      4. More about VGG
    3. GoogLeNet
      1. Inception module
      2. More about GoogLeNet
    4. Residual Networks
    5. MobileNets
      1. Depthwise separable convolution
      2. Control parameters
      3. More about MobileNets
    6. Summary
  11. Autoencoders, Variational Autoencoders, and Generative Adversarial Networks
    1. Why generative models
    2. Autoencoders
      1. Convolutional autoencoder example
      2. Uses and limitations of autoencoders
    3. Variational autoencoders
      1. Parameters to define a normal distribution
      2. VAE loss function
        1. Kullback-Leibler divergence
      3. Training the VAE
      4. The reparameterization trick
      5. Convolutional Variational Autoencoder code
      6. Generating new data
    4. Generative adversarial networks
      1. The discriminator
      2. The generator
      3. GAN loss function
        1. Generator loss
        2. Discriminator loss
        3. Putting the losses together
      4. Training the GAN
      5. Deep convolutional GAN
      6. WGAN
      7. BEGAN
      8. Conditional GANs
      9. Problems with GANs
        1. Loss interpretability
        2. Mode collapse
      10. Techniques to improve GANs' trainability
        1. Minibatch discriminator
    5. Summary
  12. Transfer Learning
    1. When?
    2. How? An overview
    3. How? Code example
      1. TensorFlow useful elements
        1. An autoencoder without the decoder
        2. Selecting layers
        3. Training only some layers
      2. Complete source
    4. Summary
  13. Machine Learning Best Practices and Troubleshooting
    1. Building Machine Learning Systems
    2. Data Preparation
      1. Split of Train/Development/Test set
      2. Mismatch of the Dev and Test set
      3. When to Change Dev/Test Set
    3. Bias and Variance
    4. Data Imbalance
      1. Collecting more data
      2. Look at your performance metric
      3. Data synthesis/Augmentation
      4. Resample Data
      5. Loss function Weighting
    5. Evaluation Metrics
    6. Code Structure best Practice
      1. Singleton Pattern
    7. Recipe for CNN creation
    8. Summary
  14. Training at Scale
    1. Storing data in TFRecords
      1. Making a TFRecord
        1. Storing encoded images
        2. Sharding
    2. Making efficient pipelines
      1. Parallel calls for map transformations
        1. Getting a batch
        2. Prefetching
        3. Tracing your graph
    3. Distributed computing in TensorFlow
      1. Model/data parallelism
        1. Synchronous/asynchronous SGD
    4. When data does not fit on one computer
      1. The advantages of NoSQL systems
        1. Installing Cassandra (Ubuntu 16.04)
        2. The CQLSH tool
        3. Creating databases, tables, and indexes
        4. Doing queries in Python
        5. Populating tables in Python
        6. Doing backups
    5. Scaling computation in the cloud
      1. EC2
        1. AMI
      2. Storage (S3)
        1. SageMaker
    6. Summary
  15. References
    1. Chapter 1
    2. Chapter 2
    3. Chapter 3
    4. Chapter 4
    5. Chapter 5
    6. Chapter 7
    7. Chapter 9
  16. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Hands-On Convolutional Neural Networks with TensorFlow
  • Author(s): Iffat Zafar, Giounona Tzanidou, Richard Burton, Nimesh Patel, Leonardo Araujo
  • Release date: August 2018
  • Publisher(s): Packt Publishing
  • ISBN: 9781789130331