Deep Learning for Computer Vision

Deep Learning for Computer Vision

by Rajalingappaa Shanmugamani
Released January 2018
Publisher(s): Packt Publishing
ISBN: 9781788295628

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Learn how to model and train advanced neural networks to implement a variety of Computer Vision tasks

About This Book

  • Train different kinds of deep learning model from scratch to solve specific problems in Computer Vision
  • Combine the power of Python, Keras, and TensorFlow to build deep learning models for object detection, image classification, similarity learning, image captioning, and more
  • Includes tips on optimizing and improving the performance of your models under various constraints

Who This Book Is For

This book is targeted at data scientists and Computer Vision practitioners who wish to apply the concepts of Deep Learning to overcome any problem related to Computer Vision. A basic knowledge of programming in Python—and some understanding of machine learning concepts—is required to get the best out of this book.

What You Will Learn

  • Set up an environment for deep learning with Python, TensorFlow, and Keras
  • Define and train a model for image and video classification
  • Use features from a pre-trained Convolutional Neural Network model for image retrieval
  • Understand and implement object detection using the real-world Pedestrian Detection scenario
  • Learn about various problems in image captioning and how to overcome them by training images and text together
  • Implement similarity matching and train a model for face recognition
  • Understand the concept of generative models and use them for image generation
  • Deploy your deep learning models and optimize them for high performance

In Detail

Deep learning has shown its power in several application areas of Artificial Intelligence, especially in Computer Vision. Computer Vision is the science of understanding and manipulating images, and finds enormous applications in the areas of robotics, automation, and so on. This book will also show you, with practical examples, how to develop Computer Vision applications by leveraging the power of deep learning.

In this book, you will learn different techniques related to object classification, object detection, image segmentation, captioning, image generation, face analysis, and more. You will also explore their applications using popular Python libraries such as TensorFlow and Keras. This book will help you master state-of-the-art, deep learning algorithms and their implementation.

Style and approach

This book will teach advanced techniques for Computer Vision, applying the deep learning model in reference to various datasets.

Table of contents

  1. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Conventions used
    4. Get in touch
      1. Reviews
  2. Getting Started
    1. Understanding deep learning
      1. Perceptron
      2. Activation functions
        1. Sigmoid
        2. The hyperbolic tangent function
        3. The Rectified Linear Unit (ReLU)
      3. Artificial neural network (ANN)
        1. One-hot encoding
        2. Softmax
        3. Cross-entropy
        4. Dropout
        5. Batch normalization
        6. L1 and L2 regularization
      4. Training neural networks
        1. Backpropagation
        2. Gradient descent
        3. Stochastic gradient descent
      5. Playing with TensorFlow playground
      6. Convolutional neural network
        1. Kernel
        2. Max pooling
      7. Recurrent neural networks (RNN)
      8. Long short-term memory (LSTM)
    2. Deep learning for computer vision
      1. Classification
      2. Detection or localization and segmentation
      3. Similarity learning
      4. Image captioning
      5. Generative models
      6. Video analysis
    3. Development environment setup
      1. Hardware and Operating Systems - OS
        1. General Purpose - Graphics Processing Unit (GP-GPU)
          1. Computer Unified Device Architecture - CUDA
          2. CUDA Deep Neural Network - CUDNN
      2. Installing software packages
        1. Python
        2. Open Computer Vision - OpenCV
        3. The TensorFlow library
          1. Installing TensorFlow
          2. TensorFlow example to print Hello, TensorFlow
          3. TensorFlow example for adding two numbers
          4. TensorBoard
          5. The TensorFlow Serving tool
        4. The Keras library
    4. Summary
  3. Image Classification
    1. Training the MNIST model in TensorFlow
      1. The MNIST datasets
      2. Loading the MNIST data
      3. Building a perceptron
        1. Defining placeholders for input data and targets
        2. Defining the variables for a fully connected layer
        3. Training the model with data
      4. Building a multilayer convolutional network
        1. Utilizing TensorBoard in deep learning
    2. Training the MNIST model in Keras
      1. Preparing the dataset
      2. Building the model
    3. Other popular image testing datasets 
      1. The CIFAR dataset
      2. The Fashion-MNIST dataset
      3. The ImageNet dataset and competition
    4. The bigger deep learning models
      1. The AlexNet model
      2. The VGG-16 model
      3. The Google Inception-V3 model
      4. The Microsoft ResNet-50 model
      5. The SqueezeNet model
      6. Spatial transformer networks
      7. The DenseNet model
    5. Training a model for cats versus dogs
      1. Preparing the data
      2. Benchmarking with simple CNN
      3. Augmenting the dataset
        1. Augmentation techniques 
      4. Transfer learning or fine-tuning of a model
        1. Training on bottleneck features
      5. Fine-tuning several layers in deep learning
    6. Developing real-world applications
      1. Choosing the right model
      2. Tackling the underfitting and overfitting scenarios
      3. Gender and age detection from face
      4. Fine-tuning apparel models 
      5. Brand safety
    7. Summary
  4. Image Retrieval
    1. Understanding visual features
      1. Visualizing activation of deep learning models
      2. Embedding visualization
        1. Guided backpropagation
      3. The DeepDream
      4. Adversarial examples
    2. Model inference
      1. Exporting a model
      2. Serving the trained model 
    3. Content-based image retrieval
      1. Building the retrieval pipeline
        1. Extracting bottleneck features for an image
        2. Computing similarity between query image and target database
      2. Efficient retrieval
        1. Matching faster using approximate nearest neighbour
          1. Advantages of ANNOY
        2. Autoencoders of raw images
      3. Denoising using autoencoders
    4. Summary
  5. Object Detection
    1. Detecting objects in an image
    2. Exploring the datasets
      1. ImageNet dataset
      2. PASCAL VOC challenge
      3. COCO object detection challenge
      4. Evaluating datasets using metrics
        1. Intersection over Union
        2. The mean average precision
    3. Localizing algorithms 
      1. Localizing objects using sliding windows
        1. The scale-space concept
        2. Training a fully connected layer as a convolution layer
        3. Convolution implementation of sliding window
      2. Thinking about localization as a regression problem
        1. Applying regression to other problems
        2. Combining regression with the sliding window
    4. Detecting objects
      1. Regions of the convolutional neural network (R-CNN)
      2. Fast R-CNN
      3. Faster R-CNN
      4. Single shot multi-box detector
    5. Object detection API
      1. Installation and setup
      2. Pre-trained models
      3. Re-training object detection models
        1. Data preparation for the Pet dataset
        2. Object detection training pipeline
        3. Training the model
        4. Monitoring loss and accuracy using TensorBoard
      4. Training a pedestrian detection for a self-driving car
    6. The YOLO object detection algorithm 
    7. Summary
  6. Semantic Segmentation
    1. Predicting pixels
      1. Diagnosing medical images
      2. Understanding the earth from satellite imagery
      3. Enabling robots to see
    2. Datasets
    3. Algorithms for semantic segmentation
      1. The Fully Convolutional Network
      2. The SegNet architecture
        1. Upsampling the layers by pooling
        2. Sampling the layers by convolution
        3. Skipping connections for better training
      3. Dilated convolutions
      4. DeepLab
      5. RefiNet
      6. PSPnet
      7. Large kernel matters
      8. DeepLab v3
    4. Ultra-nerve segmentation
    5. Segmenting satellite images
      1. Modeling FCN for segmentation
    6. Segmenting instances
    7. Summary
  7. Similarity Learning
    1. Algorithms for similarity learning
      1. Siamese networks
        1. Contrastive loss
      2. FaceNet
        1. Triplet loss
      3. The DeepNet model
      4. DeepRank
      5. Visual recommendation systems
    2. Human face analysis
      1. Face detection
      2. Face landmarks and attributes
        1. The Multi-Task Facial Landmark (MTFL) dataset
        2. The Kaggle keypoint dataset
        3. The Multi-Attribute Facial Landmark (MAFL) dataset
        4. Learning the facial key points
      3. Face recognition
        1. The labeled faces in the wild (LFW) dataset
        2. The YouTube faces dataset
        3. The CelebFaces Attributes dataset (CelebA) 
        4. CASIA web face database
        5. The VGGFace2 dataset
        6. Computing the similarity between faces
        7. Finding the optimum threshold
      4. Face clustering 
    3. Summary
  8. Image Captioning
    1. Understanding the problem and datasets
    2. Understanding natural language processing for image captioning
      1. Expressing words in vector form
      2. Converting words to vectors
      3. Training an embedding
    3. Approaches for image captioning and related problems
      1. Using a condition random field for linking image and text
      2. Using RNN on CNN features to generate captions
      3. Creating captions using image ranking
      4. Retrieving captions from images and images from captions
      5. Dense captioning 
      6. Using RNN for captioning
      7. Using multimodal metric space
      8. Using attention network for captioning
      9. Knowing when to look
    4. Implementing attention-based image captioning
    5. Summary
  9. Generative Models
    1. Applications of generative models
      1. Artistic style transfer
      2. Predicting the next frame in a video 
      3. Super-resolution of images
      4. Interactive image generation
      5. Image to image translation
      6. Text to image generation
      7. Inpainting
      8. Blending
      9. Transforming attributes
      10. Creating training data
      11. Creating new animation characters
      12. 3D models from photos
    2. Neural artistic style transfer
      1. Content loss
      2. Style loss using the Gram matrix
      3. Style transfer
    3. Generative Adversarial Networks
      1. Vanilla GAN
      2. Conditional GAN
      3. Adversarial loss
      4. Image translation
      5. InfoGAN
      6. Drawbacks of GAN
    4. Visual dialogue model
      1. Algorithm for VDM
        1. Generator
        2. Discriminator
    5. Summary
  10. Video Classification
    1. Understanding and classifying videos 
      1. Exploring video classification datasets
        1. UCF101
        2. YouTube-8M
        3. Other datasets
      2. Splitting videos into frames
      3. Approaches for classifying videos
        1. Fusing parallel CNN for video classification
        2. Classifying videos over long periods
        3. Streaming two CNN's for action recognition
        4. Using 3D convolution for temporal learning
        5. Using trajectory for classification
        6. Multi-modal fusion
        7. Attending regions for classification
    2. Extending image-based approaches to videos
      1. Regressing the human pose
        1. Tracking facial landmarks
      2. Segmenting videos
      3. Captioning videos
      4. Generating videos
    3. Summary
  11. Deployment
    1. Performance of models
      1. Quantizing the models
      2. MobileNets
    2. Deployment in the cloud
      1. AWS
      2. Google Cloud Platform
    3. Deployment of models in devices
      1. Jetson TX2
      2. Android
      3. iPhone
    4. Summary
  12. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Deep Learning for Computer Vision
  • Author(s): Rajalingappaa Shanmugamani
  • Release date: January 2018
  • Publisher(s): Packt Publishing
  • ISBN: 9781788295628