TensorFlow in Action

TensorFlow in Action

by Thushan Ganegedara
Released October 2022
Publisher(s): Manning Publications
ISBN: 9781617298349

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Unlock the TensorFlow design secrets behind successful deep learning applications! Deep learning StackOverflow contributor Thushan Ganegedara teaches you the new features of TensorFlow 2 in this hands-on guide.
In TensorFlow in Action you will learn:

  • Fundamentals of TensorFlow
  • Implementing deep learning networks
  • Picking a high-level Keras API for model building with confidence
  • Writing comprehensive end-to-end data pipelines
  • Building models for computer vision and natural language processing
  • Utilizing pretrained NLP models
  • Recent algorithms including transformers, attention models, and ElMo

In TensorFlow in Action, you'll dig into the newest version of Google's amazing TensorFlow framework as you learn to create incredible deep learning applications. Author Thushan Ganegedara uses quirky stories, practical examples, and behind-the-scenes explanations to demystify concepts otherwise trapped in dense academic papers. As you dive into modern deep learning techniques like transformer and attention models, you’ll benefit from the unique insights of a top StackOverflow contributor for deep learning and NLP.

About the Technology
Google’s TensorFlow framework sits at the heart of modern deep learning. Boasting practical features like multi-GPU support, network data visualization, and easy production pipelines using TensorFlow Extended (TFX), TensorFlow provides the most efficient path to professional AI applications. And the Keras library, fully integrated into TensorFlow 2, makes it a snap to build and train even complex models for vision, language, and more.

About the Book
TensorFlow in Action teaches you to construct, train, and deploy deep learning models using TensorFlow 2. In this practical tutorial, you’ll build reusable skill hands-on as you create production-ready applications such as a French-to-English translator and a neural network that can write fiction. You’ll appreciate the in-depth explanations that go from DL basics to advanced applications in NLP, image processing, and MLOps, complete with important details that you’ll return to reference over and over.

What's Inside
  • Covers TensorFlow 2.9
  • Recent algorithms including transformers, attention models, and ElMo
  • Build on pretrained models
  • Writing end-to-end data pipelines with TFX


About the Reader
For Python programmers with basic deep learning skills.

About the Author
Thushan Ganegedara is a senior ML engineer at Canva and TensorFlow expert. He holds a PhD in machine learning from the University of Sydney.

Quotes
A nice dive into TensorFlow 2. It outlines modern techniques and reinforces them with concrete sample projects.
- Joshua A. McAdams, Google

Practical and hands-on. A valuable resource for practitioners and newbies.
- Amaresh Rajasekharan, IBM

Comprehensive. Covers advanced topics such as atrous convolution, Transformers, and MLOps.
- Wei Luo, Deakin University

Recommended for practitioners. There are code examples for every topic.
- Vidhya Vinay, Streamingo.ai

Table of contents

  1. TensorFlow in Action
  2. Copyright
  3. dedication
  4. contents
  5. front matter
    1. preface
    2. acknowledgments
    3. about this book
      1. Who should read this book?
      2. How this book is organized: A roadmap
      3. About the code
      4. liveBook discussion forum
    4. about the author
    5. about the cover illustration
  6. Part 1 Foundations of TensorFlow 2 and deep learning
  7. 1 The amazing world of TensorFlow
    1. 1.1 What is TensorFlow?
      1. 1.1.1 An overview of popular components of TensorFlow
      2. 1.1.2 Building and deploying a machine learning model
    2. 1.2 GPU vs. CPU
    3. 1.3 When and when not to use TensorFlow
      1. 1.3.1 When to use TensorFlow
      2. 1.3.2 When not to use TensorFlow
    4. 1.4 What will this book teach you?
      1. 1.4.1 TensorFlow fundamentals
      2. 1.4.2 Deep learning algorithms
      3. 1.4.3 Monitoring and optimization
    5. 1.5 Who is this book for?
    6. 1.6 Should we really care about Python and TensorFlow 2?
    7. Summary
  8. 2 TensorFlow 2
    1. 2.1 First steps with TensorFlow 2
      1. 2.1.1 How does TensorFlow operate under the hood?
    2. 2.2 TensorFlow building blocks
      1. 2.2.1 Understanding tf.Variable
      2. 2.2.2 Understanding tf.Tensor
      3. 2.2.3 Understanding tf.Operation
    3. 2.3 Neural network-related computations in TensorFlow
      1. 2.3.1 Matrix multiplication
      2. 2.3.2 Convolution operation
      3. 2.3.3 Pooling operation
    4. Summary
    5. Answers to exercises
  9. 3 Keras and data retrieval in TensorFlow 2
    1. 3.1 Keras model-building APIs
      1. 3.1.1 Introducing the data set
      2. 3.1.2 The Sequential API
      3. 3.1.3 The functional API
      4. 3.1.4 The sub-classing API
    2. 3.2 Retrieving data for TensorFlow/Keras models
      1. 3.2.1 tf.data API
      2. 3.2.2 Keras DataGenerators
      3. 3.2.3 tensorflow-datasets package
    3. Summary
    4. Answers to exercises
  10. 4 Dipping toes in deep learning
    1. 4.1 Fully connected networks
      1. 4.1.1 Understanding the data
      2. 4.1.2 Autoencoder model
    2. 4.2 Convolutional neural networks
      1. 4.2.1 Understanding the data
      2. 4.2.2 Implementing the network
    3. 4.3 One step at a time: Recurrent neural networks (RNNs)
      1. 4.3.1 Understanding the data
      2. 4.3.2 Implementing the model
      3. 4.3.3 Predicting future CO2 values with the trained model
    4. Summary
    5. Answers to exercises
  11. 5 State-of-the-art in deep learning: Transformers
    1. 5.1 Representing text as numbers
    2. 5.2 Understanding the Transformer model
      1. 5.2.1 The encoder-decoder view of the Transformer
      2. 5.2.2 Diving deeper
      3. 5.2.3 Self-attention layer
      4. 5.2.4 Understanding self-attention using scalars
      5. 5.2.5 Self-attention as a cooking competition
      6. 5.2.6 Masked self-attention layers
      7. 5.2.7 Multi-head attention
      8. 5.2.8 Fully connected layer
      9. 5.2.9 Putting everything together
    3. Summary
    4. Answers to exercises
  12. Part 2 Look ma, no hands! Deep networks in the real world
  13. 6 Teaching machines to see: Image classification with CNNs
    1. 6.1 Putting the data under the microscope: Exploratory data analysis
      1. 6.1.1 The folder/file structure
      2. 6.1.2 Understanding the classes in the data set
      3. 6.1.3 Computing simple statistics on the data set
    2. 6.2 Creating data pipelines using the Keras ImageDataGenerator
    3. 6.3 Inception net: Implementing a state-of-the-art image classifier
      1. 6.3.1 Recap on CNNs
      2. 6.3.2 Inception net v1
      3. 6.3.3 Putting everything together
      4. 6.3.4 Other Inception models
    4. 6.4 Training the model and evaluating performance
    5. Summary
    6. Answers to exercises
  14. 7 Teaching machines to see better: Improving CNNs and making them confess
    1. 7.1 Techniques for reducing overfitting
      1. 7.1.1 Image data augmentation with Keras
      2. 7.1.2 Dropout: Randomly switching off parts of your network to improve generalizability
      3. 7.1.3 Early stopping: Halting the training process if the network starts to underperform
    2. 7.2 Toward minimalism: Minception instead of Inception
      1. 7.2.1 Implementing the stem
      2. 7.2.2 Implementing Inception-ResNet type A block
      3. 7.2.3 Implementing the Inception-ResNet type B block
      4. 7.2.4 Implementing the reduction block
      5. 7.2.5 Putting everything together
      6. 7.2.6 Training Minception
    3. 7.3 If you can't beat them, join ‘em: Using pretrained networks for enhancing performance
      1. 7.3.1 Transfer learning: Reusing existing knowledge in deep neural networks
    4. 7.4 Grad-CAM: Making CNNs confess
    5. Summary
    6. Answers to exercises
  15. 8 Telling things apart: Image segmentation
    1. 8.1 Understanding the data
    2. 8.2 Getting serious: Defining a TensorFlow data pipeline
      1. 8.2.1 Optimizing tf.data pipelines
      2. 8.2.2 The final tf.data pipeline
    3. 8.3 DeepLabv3: Using pretrained networks to segment images
      1. 8.3.1 A quick overview of the ResNet-50 model
      2. 8.3.2 Atrous convolution: Increasing the receptive field of convolution layers with holes
      3. 8.3.3 Implementing DeepLab v3 using the Keras functional API
      4. 8.3.4 Implementing the atrous spatial pyramid pooling module
      5. 8.3.5 Putting everything together
    4. 8.4 Compiling the model: Loss functions and evaluation metrics in image segmentation
      1. 8.4.1 Loss functions
      2. 8.4.2 Evaluation metrics
    5. 8.5 Training the model
    6. 8.6 Evaluating the model
    7. Summary
    8. Answers to exercises
  16. 9 Natural language processing with TensorFlow: Sentiment analysis
    1. 9.1 What the text? Exploring and processing text
    2. 9.2 Getting text ready for the model
      1. 9.2.1 Splitting training/validation and testing data
      2. 9.2.2 Analyze the vocabulary
      3. 9.2.3 Analyzing the sequence length
      4. 9.2.4 Text to words and then to numbers with Keras
    3. 9.3 Defining an end-to-end NLP pipeline with TensorFlow
    4. 9.4 Happy reviews mean happy customers: Sentiment analysis
      1. 9.4.1 LSTM Networks
      2. 9.4.2 Defining the final model
    5. 9.5 Training and evaluating the model
    6. 9.6 Injecting semantics with word vectors
      1. 9.6.1 Word embeddings
      2. 9.6.2 Defining the final model with word embeddings
      3. 9.6.3 Training and evaluating the model
    7. Summary
    8. Answers to exercises
  17. 10 Natural language processing with TensorFlow: Language modeling
    1. 10.1 Processing the data
      1. 10.1.1 What is language modeling?
      2. 10.1.2 Downloading and playing with data
      3. 10.1.3 Too large vocabulary? N-grams to the rescue
      4. 10.1.4 Tokenizing text
      5. 10.1.5 Defining a tf.data pipeline
    2. 10.2 GRUs in Wonderland: Generating text with deep learning
    3. 10.3 Measuring the quality of the generated text
    4. 10.4 Training and evaluating the language model
    5. 10.5 Generating new text from the language model: Greedy decoding
    6. 10.6 Beam search: Enhancing the predictive power of sequential models
    7. Summary
    8. Answers to exercises
  18. Part 3 Advanced deep networks for complex problems
  19. 11 Sequence-to-sequence learning: Part 1
    1. 11.1 Understanding the machine translation data
    2. 11.2 Writing an English-German seq2seq machine translator
      1. 11.2.1 The TextVectorization layer
      2. 11.2.2 Defining the TextVectorization layers for the seq2seq model
      3. 11.2.3 Defining the encoder
      4. 11.2.4 Defining the decoder and the final model
      5. 11.2.5 Compiling the model
    3. 11.3 Training and evaluating the model
    4. 11.4 From training to inference: Defining the inference model
    5. Summary
    6. Answers to exercises
  20. 12 Sequence-to-sequence learning: Part 2
    1. 12.1 Eyeballing the past: Improving our model with attention
      1. 12.1.1 Implementing Bahdanau attention in TensorFlow
      2. 12.1.2 Defining the final model
      3. 12.1.3 Training the model
    2. 12.2 Visualizing the attention
    3. Summary
    4. Answers to exercises
  21. 13 Transformers
    1. 13.1 Transformers in more detail
      1. 13.1.1 Revisiting the basic components of the Transformer
      2. 13.1.2 Embeddings in the Transformer
      3. 13.1.3 Residuals and normalization
    2. 13.2 Using pretrained BERT for spam classification
      1. 13.2.1 Understanding BERT
      2. 13.2.2 Classifying spam with BERT in TensorFlow
    3. 13.3 Question answering with Hugging Face’s Transformers
      1. 13.3.1 Understanding the data
      2. 13.3.2 Processing data
      3. 13.3.3 Defining the DistilBERT model
      4. 13.3.4 Training the model
      5. 13.3.5 Ask BERT a question
    4. Summary
    5. Answers to exercises
  22. 14 TensorBoard: Big brother of TensorFlow
    1. 14.1 Visualize data with TensorBoard
    2. 14.2 Tracking and monitoring models with TensorBoard
    3. 14.3 Using tf.summary to write custom metrics during model training
    4. 14.4 Profiling models to detect performance bottlenecks
      1. 14.4.1 Optimizing the input pipeline
      2. 14.4.2 Mixed precision training
    5. 14.5 Visualizing word vectors with the TensorBoard
    6. Summary
    7. Answers to exercises
  23. 15 TFX: MLOps and deploying models with TensorFlow
    1. 15.1 Writing a data pipeline with TFX
      1. 15.1.1 Loading data from CSV files
      2. 15.1.2 Generating basic statistics from the data
      3. 15.1.3 Inferring the schema from data
      4. 15.1.4 Converting data to features
    2. 15.2 Training a simple regression neural network: TFX Trainer API
      1. 15.2.1 Defining a Keras model
      2. 15.2.2 Defining the model training
      3. 15.2.3 SignatureDefs: Defining how models are used outside TensorFlow
      4. 15.2.4 Training the Keras model with TFX Trainer
    3. 15.3 Setting up Docker to serve a trained model
    4. 15.4 Deploying the model and serving it through an API
      1. 15.4.1 Validating the infrastructure
      2. 15.4.2 Resolving the correct model
      3. 15.4.3 Evaluating the model
      4. 15.4.4 Pushing the final model
      5. 15.4.5 Predicting with the TensorFlow serving API
    5. Summary
    6. Answers to exercises
  24. appendix A Setting up the environment
    1. A.1 In a Unix-based environment
      1. A.1.1 Creating a virtual Python environment with Anaconda distribution (Ubuntu)
      2. A.1.2 Prerequisites for GPU support (Ubuntu)
      3. A.1.3 Notes on MacOS
    2. A.2 In Windows Environments
      1. A.2.1 Creating a Virtual Python Environment (Anaconda)
      2. A.2.2 Prerequisites for GPU support
    3. A.3 Activating and deactivating the conda environment
    4. A.4 Running the Jupyter Notebook server and creating notebooks
    5. A.5 Miscellaneous notes
  25. appendix B Computer vision
    1. B.1 Grad-CAM: Interpreting computer vision models
    2. B.2 Image segmentation: U-Net model
      1. B.2.1 Understanding and defining the U-Net model
      2. B.2.2 What’s better than an encoder? A pretrained encoder
  26. appendix C Natural language processing
    1. C.1 Touring around the zoo: Meeting other Transformer models
      1. C.1.1 Generative pre-training (GPT) model (2018)
      2. C.1.2 DistilBERT (2019)
      3. C.1.3 RoBERT/ToBERT (2019)
      4. C.1.4 BART (2019)
      5. C.1.5 XLNet (2020)
      6. C.1.6 Albert (2020)
      7. C.1.7 Reformer (2020)
  27. index

Product information

  • Title: TensorFlow in Action
  • Author(s): Thushan Ganegedara
  • Release date: October 2022
  • Publisher(s): Manning Publications
  • ISBN: 9781617298349