Practical Deep Learning for Cloud, Mobile, and Edge

Practical Deep Learning for Cloud, Mobile, and Edge

by Anirudh Koul, Siddha Ganju, Meher Kasam
Released October 2019
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781492034865

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Buy on Amazon Buy on ebooks.com
Start your free trial

Book description

Whether you’re a software engineer aspiring to enter the world of deep learning, a veteran data scientist, or a hobbyist with a simple dream of making the next viral AI app, you might have wondered where to begin. This step-by-step guide teaches you how to build practical deep learning applications for the cloud, mobile, browsers, and edge devices using a hands-on approach.

Relying on years of industry experience transforming deep learning research into award-winning applications, Anirudh Koul, Siddha Ganju, and Meher Kasam guide you through the process of converting an idea into something that people in the real world can use.

  • Train, tune, and deploy computer vision models with Keras, TensorFlow, Core ML, and TensorFlow Lite
  • Develop AI for a range of devices including Raspberry Pi, Jetson Nano, and Google Coral
  • Explore fun projects, from Silicon Valley’s Not Hotdog app to 40+ industry case studies
  • Simulate an autonomous car in a video game environment and build a miniature version with reinforcement learning
  • Use transfer learning to train models in minutes
  • Discover 50+ practical tips for maximizing model accuracy and speed, debugging, and scaling to millions of users

Publisher resources

View/Submit Errata

Table of contents

  1. Preface
    1. To the Backend/Frontend/Mobile Software Developer
    2. To the Data Scientist
    3. To the Student
    4. To the Teacher
    5. To the Robotics Enthusiast
    6. What to Expect in Each Chapter
    7. Conventions Used in This Book
    8. Using Code Examples
    9. O’Reilly Online Learning
    10. How to Contact Us
    11. Acknowledgments
      1. Group Acknowledgments
      2. Personal Acknowledgments
  2. 1. Exploring the Landscape of Artificial Intelligence
    1. An Apology
    2. The Real Introduction
    3. What Is AI?
      1. Motivating Examples
    4. A Brief History of AI
      1. Exciting Beginnings
      2. The Cold and Dark Days
      3. A Glimmer of Hope
      4. How Deep Learning Became a Thing
    5. Recipe for the Perfect Deep Learning Solution
      1. Datasets
      2. Model Architecture
      3. Frameworks
      4. Hardware
    6. Responsible AI
      1. Bias
      2. Accountability and Explainability
      3. Reproducibility
      4. Robustness
      5. Privacy
    7. Summary
    8. Frequently Asked Questions
  3. 2. What’s in the Picture: Image Classification with Keras
    1. Introducing Keras
    2. Predicting an Image’s Category
    3. Investigating the Model
      1. ImageNet Dataset
      2. Model Zoos
      3. Class Activation Maps
    4. Summary
  4. 3. Cats Versus Dogs: Transfer Learning in 30 Lines with Keras
    1. Adapting Pretrained Models to New Tasks
      1. A Shallow Dive into Convolutional Neural Networks
      2. Transfer Learning
      3. Fine Tuning
      4. How Much to Fine Tune
    2. Building a Custom Classifier in Keras with Transfer Learning
    3. Organize the Data
    4. Build the Data Pipeline
      1. Number of Classes
      2. Batch Size
    5. Data Augmentation
    6. Model Definition
    7. Train the Model
      1. Set Training Parameters
      2. Start Training
    8. Test the Model
    9. Analyzing the Results
    10. Further Reading
    11. Summary
  5. 4. Building a Reverse Image Search Engine: Understanding Embeddings
    1. Image Similarity
    2. Feature Extraction
    3. Similarity Search
    4. Visualizing Image Clusters with t-SNE
    5. Improving the Speed of Similarity Search
      1. Length of Feature Vectors
      2. Reducing Feature-Length with PCA
    6. Scaling Similarity Search with Approximate Nearest Neighbors
      1. Approximate Nearest-Neighbor Benchmark
      2. Which Library Should I Use?
      3. Creating a Synthetic Dataset
      4. Brute Force
      5. Annoy
      6. NGT
      7. Faiss
    7. Improving Accuracy with Fine Tuning
      1. Fine Tuning Without Fully Connected Layers
    8. Siamese Networks for One-Shot Face Verification
    9. Case Studies
      1. Flickr
      2. Pinterest
      3. Celebrity Doppelgangers
      4. Spotify
      5. Image Captioning
    10. Summary
  6. 5. From Novice to Master Predictor: Maximizing Convolutional Neural Network Accuracy
    1. Tools of the Trade
      1. TensorFlow Datasets
      2. TensorBoard
      3. What-If Tool
      4. tf-explain
    2. Common Techniques for Machine Learning Experimentation
      1. Data Inspection
      2. Breaking the Data: Train, Validation, Test
      3. Early Stopping
      4. Reproducible Experiments
    3. End-to-End Deep Learning Example Pipeline
      1. Basic Transfer Learning Pipeline
      2. Basic Custom Network Pipeline
    4. How Hyperparameters Affect Accuracy
      1. Transfer Learning Versus Training from Scratch
      2. Effect of Number of Layers Fine-Tuned in Transfer Learning
      3. Effect of Data Size on Transfer Learning
      4. Effect of Learning Rate
      5. Effect of Optimizers
      6. Effect of Batch Size
      7. Effect of Resizing
      8. Effect of Change in Aspect Ratio on Transfer Learning
    5. Tools to Automate Tuning for Maximum Accuracy
      1. Keras Tuner
      2. AutoAugment
      3. AutoKeras
    6. Summary
  7. 6. Maximizing Speed and Performance of TensorFlow: A Handy Checklist
    1. GPU Starvation
      1. nvidia-smi
      2. TensorFlow Profiler + TensorBoard
    2. How to Use This Checklist
    3. Performance Checklist
      1. Data Preparation
      2. Data Reading
      3. Data Augmentation
      4. Training
      5. Inference
    4. Data Preparation
      1. Store as TFRecords
      2. Reduce Size of Input Data
      3. Use TensorFlow Datasets
    5. Data Reading
      1. Use tf.data
      2. Prefetch Data
      3. Parallelize CPU Processing
      4. Parallelize I/O and Processing
      5. Enable Nondeterministic Ordering
      6. Cache Data
      7. Turn on Experimental Optimizations
      8. Autotune Parameter Values
    6. Data Augmentation
      1. Use GPU for Augmentation
    7. Training
      1. Use Automatic Mixed Precision
      2. Use Larger Batch Size
      3. Use Multiples of Eight
      4. Find the Optimal Learning Rate
      5. Use tf.function
      6. Overtrain, and Then Generalize
      7. Install an Optimized Stack for the Hardware
      8. Optimize the Number of Parallel CPU Threads
      9. Use Better Hardware
      10. Distribute Training
      11. Examine Industry Benchmarks
    8. Inference
      1. Use an Efficient Model
      2. Quantize the Model
      3. Prune the Model
      4. Use Fused Operations
      5. Enable GPU Persistence
    9. Summary
  8. 7. Practical Tools, Tips, and Tricks
    1. Installation
    2. Training
    3. Model
    4. Data
    5. Privacy
    6. Education and Exploration
    7. One Last Question
  9. 8. Cloud APIs for Computer Vision: Up and Running in 15 Minutes
    1. The Landscape of Visual Recognition APIs
      1. Clarifai
      2. Microsoft Cognitive Services
      3. Google Cloud Vision
      4. Amazon Rekognition
      5. IBM Watson Visual Recognition
      6. Algorithmia
    2. Comparing Visual Recognition APIs
      1. Service Offerings
      2. Cost
      3. Accuracy
      4. Bias
    3. Getting Up and Running with Cloud APIs
    4. Training Our Own Custom Classifier
      1. Top Reasons Why Our Classifier Does Not Work Satisfactorily
    5. Comparing Custom Classification APIs
    6. Performance Tuning for Cloud APIs
      1. Effect of Resizing on Image Labeling APIs
      2. Effect of Compression on Image Labeling APIs
      3. Effect of Compression on OCR APIs
      4. Effect of Resizing on OCR APIs
    7. Case Studies
      1. The New York Times
      2. Uber
      3. Giphy
      4. OmniEarth
      5. Photobucket
      6. Staples
      7. InDro Robotics
    8. Summary
  10. 9. Scalable Inference Serving on Cloud with TensorFlow Serving and KubeFlow
    1. Landscape of Serving AI Predictions
    2. Flask: Build Your Own Server
      1. Making a REST API with Flask
      2. Deploying a Keras Model to Flask
      3. Pros of Using Flask
      4. Cons of Using Flask
    3. Desirable Qualities in a Production-Level Serving System
      1. High Availability
      2. Scalability
      3. Low Latency
      4. Geographic Availability
      5. Failure Handling
      6. Monitoring
      7. Model Versioning
      8. A/B Testing
      9. Support for Multiple Machine Learning Libraries
    4. Google Cloud ML Engine: A Managed Cloud AI Serving Stack
      1. Pros of Using Cloud ML Engine
      2. Cons of Using Cloud ML Engine
      3. Building a Classification API
    5. TensorFlow Serving
      1. Installation
    6. KubeFlow
      1. Pipelines
      2. Fairing
      3. Installation
    7. Price Versus Performance Considerations
      1. Cost Analysis of Inference-as-a-Service
      2. Cost Analysis of Building Your Own Stack
    8. Summary
  11. 10. AI in the Browser with TensorFlow.js and ml5.js
    1. JavaScript-Based Machine Learning Libraries: A Brief History
      1. ConvNetJS
      2. Keras.js
      3. ONNX.js
      4. TensorFlow.js
    2. TensorFlow.js Architecture
    3. Running Pretrained Models Using TensorFlow.js
    4. Model Conversion for the Browser
    5. Training in the Browser
      1. Feature Extraction
      2. Data Collection
      3. Training
      4. GPU Utilization
    6. ml5.js
    7. PoseNet
    8. pix2pix
    9. Benchmarking and Practical Considerations
      1. Model Size
      2. Inference Time
    10. Case Studies
      1. Semi-Conductor
      2. TensorSpace
      3. Metacar
      4. Airbnb’s Photo Classification
      5. GAN Lab
    11. Summary
  12. 11. Real-Time Object Classification on iOS with Core ML
    1. The Development Life Cycle for Artificial Intelligence on Mobile
    2. A Brief History of Core ML
    3. Alternatives to Core ML
      1. TensorFlow Lite
      2. ML Kit
      3. Fritz
    4. Apple’s Machine Learning Architecture
      1. Domain-Based Frameworks
      2. ML Framework
      3. ML Performance Primitives
    5. Building a Real-Time Object Recognition App
    6. Conversion to Core ML
      1. Conversion from Keras
      2. Conversion from TensorFlow
    7. Dynamic Model Deployment
    8. On-Device Training
      1. Federated Learning
    9. Performance Analysis
      1. Benchmarking Models on iPhones
    10. Measuring Energy Impact
      1. Benchmarking Load
    11. Reducing App Size
      1. Avoid Bundling the Model
      2. Use Quantization
      3. Use Create ML
    12. Case Studies
      1. Magic Sudoku
      2. Seeing AI
      3. HomeCourt
      4. InstaSaber + YoPuppet
    13. Summary
  13. 12. Not Hotdog on iOS with Core ML and Create ML
    1. Collecting Data
      1. Approach 1: Find or Collect a Dataset
      2. Approach 2: Fatkun Chrome Browser Extension
      3. Approach 3: Web Scraper Using Bing Image Search API
    2. Training Our Model
      1. Approach 1: Use Web UI-based Tools
      2. Approach 2: Use Create ML
      3. Approach 3: Fine Tuning Using Keras
    3. Model Conversion Using Core ML Tools
    4. Building the iOS App
    5. Further Exploration
    6. Summary
  14. 13. Shazam for Food: Developing Android Apps with TensorFlow Lite and ML Kit
    1. The Life Cycle of a Food Classifier App
    2. An Overview of TensorFlow Lite
      1. TensorFlow Lite Architecture
    3. Model Conversion to TensorFlow Lite
    4. Building a Real-Time Object Recognition App
    5. ML Kit + Firebase
      1. Object Classification in ML Kit
      2. Custom Models in ML Kit
      3. Hosted Models
      4. A/B Testing Hosted Models
      5. Using the Experiment in Code
    6. TensorFlow Lite on iOS
    7. Performance Optimizations
      1. Quantizing with TensorFlow Lite Converter
      2. TensorFlow Model Optimization Toolkit
    8. Fritz
    9. A Holistic Look at the Mobile AI App Development Cycle
      1. How Do I Collect Initial Data?
      2. How Do I Label My Data?
      3. How Do I Train My Model?
      4. How Do I Convert the Model to a Mobile-Friendly Format?
      5. How Do I Make my Model Performant?
      6. How Do I Build a Great UX for My Users?
      7. How Do I Make the Model Available to My Users?
      8. How Do I Measure the Success of My Model?
      9. How Do I Improve My Model?
      10. How Do I Update the Model on My Users’ Phones?
    10. The Self-Evolving Model
    11. Case Studies
      1. Lose It!
      2. Portrait Mode on Pixel 3 Phones
      3. Speaker Recognition by Alibaba
      4. Face Contours in ML Kit
      5. Real-Time Video Segmentation in YouTube Stories
    12. Summary
  15. 14. Building the Purrfect Cat Locator App with TensorFlow Object Detection API
    1. Types of Computer-Vision Tasks
      1. Classification
      2. Localization
      3. Detection
      4. Segmentation
    2. Approaches to Object Detection
    3. Invoking Prebuilt Cloud-Based Object Detection APIs
    4. Reusing a Pretrained Model
      1. Obtaining the Model
      2. Test Driving Our Model
      3. Deploying to a Device
    5. Building a Custom Detector Without Any Code
    6. The Evolution of Object Detection
      1. Performance Considerations
    7. Key Terms in Object Detection
      1. Intersection over Union
      2. Mean Average Precision
      3. Non-Maximum Suppression
    8. Using the TensorFlow Object Detection API to Build Custom Models
      1. Data Collection
      2. Labeling the Data
      3. Preprocessing the Data
    9. Inspecting the Model
      1. Training
      2. Model Conversion
    10. Image Segmentation
    11. Case Studies
      1. Smart Refrigerator
      2. Crowd Counting
      3. Face Detection in Seeing AI
      4. Autonomous Cars
    12. Summary
  16. 15. Becoming a Maker: Exploring Embedded AI at the Edge
    1. Exploring the Landscape of Embedded AI Devices
      1. Raspberry Pi
      2. Intel Movidius Neural Compute Stick
      3. Google Coral USB Accelerator
      4. NVIDIA Jetson Nano
      5. FPGA + PYNQ
      6. Arduino
    2. A Qualitative Comparison of Embedded AI Devices
    3. Hands-On with the Raspberry Pi
    4. Speeding Up with the Google Coral USB Accelerator
    5. Port to NVIDIA Jetson Nano
    6. Comparing the Performance of Edge Devices
    7. Case Studies
      1. JetBot
      2. Squatting for Metro Tickets
      3. Cucumber Sorter
    8. Further Exploration
    9. Summary
  17. 16. Simulating a Self-Driving Car Using End-to-End Deep Learning with Keras
    1. A Brief History of Autonomous Driving
    2. Deep Learning, Autonomous Driving, and the Data Problem
    3. The “Hello, World!” of Autonomous Driving: Steering Through a Simulated Environment
      1. Setup and Requirements
    4. Data Exploration and Preparation
      1. Identifying the Region of Interest
      2. Data Augmentation
      3. Dataset Imbalance and Driving Strategies
    5. Training Our Autonomous Driving Model
      1. Drive Data Generator
      2. Model Definition
    6. Deploying Our Autonomous Driving Model
    7. Further Exploration
      1. Expanding Our Dataset
      2. Training on Sequential Data
      3. Reinforcement Learning
    8. Summary
  18. 17. Building an Autonomous Car in Under an Hour: Reinforcement Learning with AWS DeepRacer
    1. A Brief Introduction to Reinforcement Learning
    2. Why Learn Reinforcement Learning with an Autonomous Car?
    3. Practical Deep Reinforcement Learning with DeepRacer
      1. Building Our First Reinforcement Learning
      2. Step 1: Create Model
      3. Step 2: Configure Training
      4. Step 3: Model Training
      5. Step 4: Evaluating the Performance of the Model
    4. Reinforcement Learning in Action
      1. How Does a Reinforcement Learning System Learn?
      2. Reinforcement Learning Theory
      3. Reinforcement Learning Algorithm in AWS DeepRacer
      4. Deep Reinforcement Learning Summary with DeepRacer as an Example
      5. Step 5: Improving Reinforcement Learning Models
    5. Racing the AWS DeepRacer Car
      1. Building the Track
      2. AWS DeepRacer Single-Turn Track Template
      3. Running the Model on AWS DeepRacer
      4. Driving the AWS DeepRacer Vehicle Autonomously
    6. Further Exploration
      1. DeepRacer League
      2. Advanced AWS DeepRacer
      3. AI Driving Olympics
      4. DIY Robocars
      5. Roborace
    7. Summary
  19. A Crash Course in Convolutional Neural Networks
    1. Machine Learning
    2. Perceptron
    3. Activation Functions
    4. Neural Networks
    5. Backpropagation
    6. Shortcoming of Neural Networks
    7. Desired Properties of an Image Classifier
    8. Convolution
    9. Pooling
    10. Structure of a CNN
    11. Further Exploration
  20. Index

Product information

  • Title: Practical Deep Learning for Cloud, Mobile, and Edge
  • Author(s): Anirudh Koul, Siddha Ganju, Meher Kasam
  • Release date: October 2019
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781492034865