Deep Learning with Keras

Deep Learning with Keras

by Antonio Gulli, Sujit Pal
Released April 2017
Publisher(s): Packt Publishing
ISBN: 9781787128422

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Get to grips with the basics of Keras to implement fast and efficient deep-learning models

About This Book

  • Implement various deep-learning algorithms in Keras and see how deep-learning can be used in games
  • See how various deep-learning models and practical use-cases can be implemented using Keras
  • A practical, hands-on guide with real-world examples to give you a strong foundation in Keras

Who This Book Is For

If you are a data scientist with experience in machine learning or an AI programmer with some exposure to neural networks, you will find this book a useful entry point to deep-learning with Keras. A knowledge of Python is required for this book.

What You Will Learn

  • Optimize step-by-step functions on a large neural network using the Backpropagation Algorithm
  • Fine-tune a neural network to improve the quality of results
  • Use deep learning for image and audio processing
  • Use Recursive Neural Tensor Networks (RNTNs) to outperform standard word embedding in special cases
  • Identify problems for which Recurrent Neural Network (RNN) solutions are suitable
  • Explore the process required to implement Autoencoders
  • Evolve a deep neural network using reinforcement learning

In Detail

This book starts by introducing you to supervised learning algorithms such as simple linear regression, the classical multilayer perceptron and more sophisticated deep convolutional networks. You will also explore image processing with recognition of hand written digit images, classification of images into different categories, and advanced objects recognition with related image annotations. An example of identification of salient points for face detection is also provided. Next you will be introduced to Recurrent Networks, which are optimized for processing sequence data such as text, audio or time series. Following that, you will learn about unsupervised learning algorithms such as Autoencoders and the very popular Generative Adversarial Networks (GAN). You will also explore non-traditional uses of neural networks as Style Transfer.

Finally, you will look at Reinforcement Learning and its application to AI game playing, another popular direction of research and application of neural networks.

Style and approach

This book is an easy-to-follow guide full of examples and real-world applications to help you gain an in-depth understanding of Keras. This book will showcase more than twenty working Deep Neural Networks coded in Python using Keras.

Table of contents

  1. Preface
    1. Mission
    2. How deep learning is different from machine learning and artificial intelligence
    3. What this book covers
    4. What you need for this book
    5. Who this book is for
    6. Conventions
    7. Reader feedback
    8. Customer support
      1. Downloading the example code
      2. Downloading the color images of this book
      3. Errata
      4. Piracy
      5. Questions
  2. Neural Networks Foundations
    1. Perceptron
      1. The first example of Keras code
    2. Multilayer perceptron — the first example of a network
      1. Problems in training the perceptron and a solution
      2. Activation function — sigmoid
      3. Activation function — ReLU
      4. Activation functions
    3. A real example — recognizing handwritten digits
      1. One-hot encoding — OHE
      2. Defining a simple neural net in Keras
      3. Running a simple Keras net and establishing a baseline
      4. Improving the simple net in Keras with hidden layers
      5. Further improving the simple net in Keras with dropout
      6. Testing different optimizers in Keras
      7. Increasing the number of epochs
      8. Controlling the optimizer learning rate
      9. Increasing the number of internal hidden neurons
      10. Increasing the size of batch computation
      11. Summarizing the experiments run for recognizing handwritten charts
      12. Adopting regularization for avoiding overfitting
      13. Hyperparameters tuning
      14. Predicting output
    4. A practical overview of backpropagation
    5. Towards a deep learning approach
    6. Summary
  3. Keras Installation and API
    1. Installing Keras
      1. Step 1 — install some useful dependencies
      2. Step 2 — install Theano
      3. Step 3 — install TensorFlow
      4. Step 4 — install Keras
      5. Step 5 — testing Theano, TensorFlow, and Keras
    2. Configuring Keras
    3. Installing Keras on Docker
    4. Installing Keras on Google Cloud ML
    5. Installing Keras on Amazon AWS
    6. Installing Keras on Microsoft Azure
    7. Keras API
      1. Getting started with Keras architecture
        1. What is a tensor?
        2. Composing models in Keras
          1. Sequential composition
          2. Functional composition
      2. An overview of predefined neural network layers
        1. Regular dense
        2. Recurrent neural networks — simple, LSTM, and GRU
        3. Convolutional and pooling layers
        4. Regularization
        5. Batch normalization
      3. An overview of predefined activation functions
      4. An overview of losses functions
      5. An overview of metrics
      6. An overview of optimizers
      7. Some useful operations
      8. Saving and loading the weights and the architecture of a model
    8. Callbacks for customizing the training process
      1. Checkpointing
      2. Using TensorBoard and Keras
      3. Using Quiver and Keras
    9. Summary
  4. Deep Learning with ConvNets
    1. Deep convolutional neural network — DCNN
      1. Local receptive fields
      2. Shared weights and bias
      3. Pooling layers
        1. Max-pooling
        2. Average pooling
        3. ConvNets summary
    2. An example of DCNN — LeNet
      1. LeNet code in Keras
      2. Understanding the power of deep learning
    3. Recognizing CIFAR-10 images with deep learning
      1. Improving the CIFAR-10 performance with deeper a network
      2. Improving the CIFAR-10 performance with data augmentation
      3. Predicting with CIFAR-10
    4. Very deep convolutional networks for large-scale image recognition
      1. Recognizing cats with a VGG-16 net
      2. Utilizing Keras built-in VGG-16 net module
      3. Recycling pre-built deep learning models for extracting features
      4. Very deep inception-v3 net used for transfer learning
    5. Summary
  5. Generative Adversarial Networks and WaveNet
    1. What is a GAN?
      1. Some GAN applications
    2. Deep convolutional generative adversarial networks
    3. Keras adversarial GANs for forging MNIST
    4. Keras adversarial GANs for forging CIFAR
    5. WaveNet — a generative model for learning how to produce audio
    6. Summary
  6. Word Embeddings
    1. Distributed representations
    2. word2vec
      1. The skip-gram word2vec model
      2. The CBOW word2vec model
      3. Extracting word2vec embeddings from the model
      4. Using third-party implementations of word2vec
    3. Exploring GloVe
    4. Using pre-trained embeddings
      1. Learn embeddings from scratch
      2. Fine-tuning learned embeddings from word2vec
      3. Fine-tune learned embeddings from GloVe
      4. Look up embeddings
    5. Summary
  7. Recurrent Neural Network — RNN
    1. SimpleRNN cells
      1. SimpleRNN with Keras — generating text
    2. RNN topologies
    3. Vanishing and exploding gradients
    4. Long short term memory — LSTM
      1. LSTM with Keras — sentiment analysis
    5. Gated recurrent unit — GRU
      1. GRU with Keras — POS tagging
    6. Bidirectional RNNs
    7. Stateful RNNs
      1. Stateful LSTM with Keras — predicting electricity consumption
    8. Other RNN variants
    9. Summary
  8. Additional Deep Learning Models
    1. Keras functional API
    2. Regression networks
      1. Keras regression example — predicting benzene levels in the air
    3. Unsupervised learning — autoencoders
      1. Keras autoencoder example — sentence vectors
    4. Composing deep networks
      1. Keras example — memory network for question answering
    5. Customizing Keras
      1. Keras example — using the lambda layer
      2. Keras example — building a custom normalization layer
    6. Generative models
      1. Keras example — deep dreaming
      2. Keras example — style transfer
    7. Summary
  9. AI Game Playing
    1. Reinforcement learning
      1. Maximizing future rewards
      2. Q-learning
      3. The deep Q-network as a Q-function
      4. Balancing exploration with exploitation
      5. Experience replay, or the value of experience
    2. Example - Keras deep Q-network for catch
    3. The road ahead
    4. Summary
  10. Conclusion
    1. Keras 2.0 — what is new
      1. Installing Keras 2.0
      2. API changes

Product information

  • Title: Deep Learning with Keras
  • Author(s): Antonio Gulli, Sujit Pal
  • Release date: April 2017
  • Publisher(s): Packt Publishing
  • ISBN: 9781787128422