Python Machine Learning - Third Edition

Python Machine Learning - Third Edition

by Sebastian Raschka, Vahid Mirjalili
Released December 2019
Publisher(s): Packt Publishing
ISBN: 9781789955750

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Applied machine learning with a solid foundation in theory. Revised and expanded for TensorFlow 2, GANs, and reinforcement learning.

Purchase of the print or Kindle book includes a free eBook in the PDF format.

Key Features

  • Third edition of the bestselling, widely acclaimed Python machine learning book
  • Clear and intuitive explanations take you deep into the theory and practice of Python machine learning
  • Fully updated and expanded to cover TensorFlow 2, Generative Adversarial Network models, reinforcement learning, and best practices

Book Description

Python Machine Learning, Third Edition is a comprehensive guide to machine learning and deep learning with Python. It acts as both a step-by-step tutorial, and a reference you'll keep coming back to as you build your machine learning systems.

Packed with clear explanations, visualizations, and working examples, the book covers all the essential machine learning techniques in depth. While some books teach you only to follow instructions, with this machine learning book, Raschka and Mirjalili teach the principles behind machine learning, allowing you to build models and applications for yourself.

Updated for TensorFlow 2.0, this new third edition introduces readers to its new Keras API features, as well as the latest additions to scikit-learn. It's also expanded to cover cutting-edge reinforcement learning techniques based on deep learning, as well as an introduction to GANs. Finally, this book also explores a subfield of natural language processing (NLP) called sentiment analysis, helping you learn how to use machine learning algorithms to classify documents.

This book is your companion to machine learning with Python, whether you're a Python developer new to machine learning or want to deepen your knowledge of the latest developments.

What you will learn

  • Master the frameworks, models, and techniques that enable machines to 'learn' from data
  • Use scikit-learn for machine learning and TensorFlow for deep learning
  • Apply machine learning to image classification, sentiment analysis, intelligent web applications, and more
  • Build and train neural networks, GANs, and other models
  • Discover best practices for evaluating and tuning models
  • Predict continuous target outcomes using regression analysis
  • Dig deeper into textual and social media data using sentiment analysis

Who this book is for

If you know some Python and you want to use machine learning and deep learning, pick up this book. Whether you want to start from scratch or extend your machine learning knowledge, this is an essential resource. Written for developers and data scientists who want to create practical machine learning and deep learning code, this book is ideal for anyone who wants to teach computers how to learn from data.

Table of contents

  1. Preface
    1. Get started with machine learning
      1. Practice and theory
      2. Why Python?
      3. Explore the machine learning field
    2. Who this book is for
    3. What this book covers
    4. What you need for this book
    5. To get the most out of this book
      1. Download the example code files
      2. Download the color images
      3. Conventions used
    6. Get in touch
      1. Reviews
  2. Giving Computers the Ability to Learn from Data
    1. Building intelligent machines to transform data into knowledge
    2. The three different types of machine learning
      1. Making predictions about the future with supervised learning
        1. Classification for predicting class labels
        2. Regression for predicting continuous outcomes
      2. Solving interactive problems with reinforcement learning
      3. Discovering hidden structures with unsupervised learning
        1. Finding subgroups with clustering
        2. Dimensionality reduction for data compression
    3. Introduction to the basic terminology and notations
      1. Notation and conventions used in this book
      2. Machine learning terminology
    4. A roadmap for building machine learning systems
      1. Preprocessing – getting data into shape
      2. Training and selecting a predictive model
      3. Evaluating models and predicting unseen data instances
    5. Using Python for machine learning
      1. Installing Python and packages from the Python Package Index
      2. Using the Anaconda Python distribution and package manager
      3. Packages for scientific computing, data science, and machine learning
    6. Summary
  3. Training Simple Machine Learning Algorithms for Classification
    1. Artificial neurons – a brief glimpse into the early history of machine learning
      1. The formal definition of an artificial neuron
      2. The perceptron learning rule
    2. Implementing a perceptron learning algorithm in Python
      1. An object-oriented perceptron API
      2. Training a perceptron model on the Iris dataset
    3. Adaptive linear neurons and the convergence of learning
      1. Minimizing cost functions with gradient descent
      2. Implementing Adaline in Python
      3. Improving gradient descent through feature scaling
      4. Large-scale machine learning and stochastic gradient descent
    4. Summary
  4. A Tour of Machine Learning Classifiers Using scikit-learn
    1. Choosing a classification algorithm
    2. First steps with scikit-learn – training a perceptron
    3. Modeling class probabilities via logistic regression
      1. Logistic regression and conditional probabilities
      2. Learning the weights of the logistic cost function
      3. Converting an Adaline implementation into an algorithm for logistic regression
      4. Training a logistic regression model with scikit-learn
      5. Tackling overfitting via regularization
    4. Maximum margin classification with support vector machines
      1. Maximum margin intuition
      2. Dealing with a nonlinearly separable case using slack variables
      3. Alternative implementations in scikit-learn
    5. Solving nonlinear problems using a kernel SVM
      1. Kernel methods for linearly inseparable data
      2. Using the kernel trick to find separating hyperplanes in a high-dimensional space
    6. Decision tree learning
      1. Maximizing IG – getting the most bang for your buck
      2. Building a decision tree
      3. Combining multiple decision trees via random forests
    7. K-nearest neighbors – a lazy learning algorithm
    8. Summary
  5. Building Good Training Datasets – Data Preprocessing
    1. Dealing with missing data
      1. Identifying missing values in tabular data
      2. Eliminating training examples or features with missing values
      3. Imputing missing values
      4. Understanding the scikit-learn estimator API
    2. Handling categorical data
      1. Categorical data encoding with pandas
      2. Mapping ordinal features
      3. Encoding class labels
      4. Performing one-hot encoding on nominal features
    3. Partitioning a dataset into separate training and test datasets
    4. Bringing features onto the same scale
    5. Selecting meaningful features
      1. L1 and L2 regularization as penalties against model complexity
      2. A geometric interpretation of L2 regularization
      3. Sparse solutions with L1 regularization
      4. Sequential feature selection algorithms
    6. Assessing feature importance with random forests
    7. Summary
  6. Compressing Data via Dimensionality Reduction
    1. Unsupervised dimensionality reduction via principal component analysis
      1. The main steps behind principal component analysis
      2. Extracting the principal components step by step
      3. Total and explained variance
      4. Feature transformation
      5. Principal component analysis in scikit-learn
    2. Supervised data compression via linear discriminant analysis
      1. Principal component analysis versus linear discriminant analysis
      2. The inner workings of linear discriminant analysis
      3. Computing the scatter matrices
      4. Selecting linear discriminants for the new feature subspace
      5. Projecting examples onto the new feature space
      6. LDA via scikit-learn
    3. Using kernel principal component analysis for nonlinear mappings
      1. Kernel functions and the kernel trick
      2. Implementing a kernel principal component analysis in Python
        1. Example 1 – separating half-moon shapes
        2. Example 2 – separating concentric circles
      3. Projecting new data points
      4. Kernel principal component analysis in scikit-learn
    4. Summary
  7. Learning Best Practices for Model Evaluation and Hyperparameter Tuning
    1. Streamlining workflows with pipelines
      1. Loading the Breast Cancer Wisconsin dataset
      2. Combining transformers and estimators in a pipeline
    2. Using k-fold cross-validation to assess model performance
      1. The holdout method
      2. K-fold cross-validation
    3. Debugging algorithms with learning and validation curves
      1. Diagnosing bias and variance problems with learning curves
      2. Addressing over- and underfitting with validation curves
    4. Fine-tuning machine learning models via grid search
      1. Tuning hyperparameters via grid search
      2. Algorithm selection with nested cross-validation
    5. Looking at different performance evaluation metrics
      1. Reading a confusion matrix
      2. Optimizing the precision and recall of a classification model
      3. Plotting a receiver operating characteristic
      4. Scoring metrics for multiclass classification
      5. Dealing with class imbalance
    6. Summary
  8. Combining Different Models for Ensemble Learning
    1. Learning with ensembles
    2. Combining classifiers via majority vote
      1. Implementing a simple majority vote classifier
      2. Using the majority voting principle to make predictions
      3. Evaluating and tuning the ensemble classifier
    3. Bagging – building an ensemble of classifiers from bootstrap samples
      1. Bagging in a nutshell
      2. Applying bagging to classify examples in the Wine dataset
    4. Leveraging weak learners via adaptive boosting
      1. How boosting works
      2. Applying AdaBoost using scikit-learn
    5. Summary
  9. Applying Machine Learning to Sentiment Analysis
    1. Preparing the IMDb movie review data for text processing
      1. Obtaining the movie review dataset
      2. Preprocessing the movie dataset into a more convenient format
    2. Introducing the bag-of-words model
      1. Transforming words into feature vectors
      2. Assessing word relevancy via term frequency-inverse document frequency
      3. Cleaning text data
      4. Processing documents into tokens
    3. Training a logistic regression model for document classification
    4. Working with bigger data – online algorithms and out-of-core learning
    5. Topic modeling with Latent Dirichlet Allocation
      1. Decomposing text documents with LDA
      2. LDA with scikit-learn
    6. Summary
  10. Embedding a Machine Learning Model into a Web Application
    1. Serializing fitted scikit-learn estimators
    2. Setting up an SQLite database for data storage
    3. Developing a web application with Flask
      1. Our first Flask web application
      2. Form validation and rendering
        1. Setting up the directory structure
        2. Implementing a macro using the Jinja2 templating engine
        3. Adding style via CSS
        4. Creating the result page
    4. Turning the movie review classifier into a web application
      1. Files and folders – looking at the directory tree
      2. Implementing the main application as app.py
      3. Setting up the review form
      4. Creating a results page template
    5. Deploying the web application to a public server
      1. Creating a PythonAnywhere account
      2. Uploading the movie classifier application
      3. Updating the movie classifier
    6. Summary
  11. Predicting Continuous Target Variables with Regression Analysis
    1. Introducing linear regression
      1. Simple linear regression
      2. Multiple linear regression
    2. Exploring the Housing dataset
      1. Loading the Housing dataset into a data frame
      2. Visualizing the important characteristics of a dataset
      3. Looking at relationships using a correlation matrix
    3. Implementing an ordinary least squares linear regression model
      1. Solving regression for regression parameters with gradient descent
      2. Estimating the coefficient of a regression model via scikit-learn
    4. Fitting a robust regression model using RANSAC
    5. Evaluating the performance of linear regression models
    6. Using regularized methods for regression
    7. Turning a linear regression model into a curve – polynomial regression
      1. Adding polynomial terms using scikit-learn
      2. Modeling nonlinear relationships in the Housing dataset
    8. Dealing with nonlinear relationships using random forests
      1. Decision tree regression
      2. Random forest regression
    9. Summary
  12. Working with Unlabeled Data – Clustering Analysis
    1. Grouping objects by similarity using k-means
      1. K-means clustering using scikit-learn
      2. A smarter way of placing the initial cluster centroids using k-means++
      3. Hard versus soft clustering
      4. Using the elbow method to find the optimal number of clusters
      5. Quantifying the quality of clustering via silhouette plots
    2. Organizing clusters as a hierarchical tree
      1. Grouping clusters in bottom-up fashion
      2. Performing hierarchical clustering on a distance matrix
      3. Attaching dendrograms to a heat map
      4. Applying agglomerative clustering via scikit-learn
    3. Locating regions of high density via DBSCAN
    4. Summary
  13. Implementing a Multilayer Artificial Neural Network from Scratch
    1. Modeling complex functions with artificial neural networks
      1. Single-layer neural network recap
      2. Introducing the multilayer neural network architecture
      3. Activating a neural network via forward propagation
    2. Classifying handwritten digits
      1. Obtaining and preparing the MNIST dataset
      2. Implementing a multilayer perceptron
    3. Training an artificial neural network
      1. Computing the logistic cost function
      2. Developing your understanding of backpropagation
      3. Training neural networks via backpropagation
    4. About the convergence in neural networks
    5. A few last words about the neural network implementation
    6. Summary
  14. Parallelizing Neural Network Training with TensorFlow
    1. TensorFlow and training performance
      1. Performance challenges
      2. What is TensorFlow?
      3. How we will learn TensorFlow
    2. First steps with TensorFlow
      1. Installing TensorFlow
      2. Creating tensors in TensorFlow
      3. Manipulating the data type and shape of a tensor
      4. Applying mathematical operations to tensors
      5. Split, stack, and concatenate tensors
    3. Building input pipelines using tf.data – the TensorFlow Dataset API
      1. Creating a TensorFlow Dataset from existing tensors
      2. Combining two tensors into a joint dataset
      3. Shuffle, batch, and repeat
      4. Creating a dataset from files on your local storage disk
      5. Fetching available datasets from the tensorflow_datasets library
    4. Building an NN model in TensorFlow
      1. The TensorFlow Keras API (tf.keras)
      2. Building a linear regression model
      3. Model training via the .compile() and .fit() methods
      4. Building a multilayer perceptron for classifying flowers in the Iris dataset
      5. Evaluating the trained model on the test dataset
      6. Saving and reloading the trained model
    5. Choosing activation functions for multilayer neural networks
      1. Logistic function recap
      2. Estimating class probabilities in multiclass classification via the softmax function
      3. Broadening the output spectrum using a hyperbolic tangent
      4. Rectified linear unit activation
    6. Summary
  15. Going Deeper – The Mechanics of TensorFlow
    1. The key features of TensorFlow
    2. TensorFlow's computation graphs: migrating to TensorFlow v2
      1. Understanding computation graphs
      2. Creating a graph in TensorFlow v1.x
      3. Migrating a graph to TensorFlow v2
      4. Loading input data into a model: TensorFlow v1.x style
      5. Loading input data into a model: TensorFlow v2 style
      6. Improving computational performance with function decorators
    3. TensorFlow Variable objects for storing and updating model parameters
    4. Computing gradients via automatic differentiation and GradientTape
      1. Computing the gradients of the loss with respect to trainable variables
      2. Computing gradients with respect to non-trainable tensors
      3. Keeping resources for multiple gradient computations
    5. Simplifying implementations of common architectures via the Keras API
      1. Solving an XOR classification problem
      2. Making model building more flexible with Keras' functional API
      3. Implementing models based on Keras' Model class
      4. Writing custom Keras layers
    6. TensorFlow Estimators
      1. Working with feature columns
      2. Machine learning with pre-made Estimators
      3. Using Estimators for MNIST handwritten digit classification
      4. Creating a custom Estimator from an existing Keras model
    7. Summary
  16. Classifying Images with Deep Convolutional Neural Networks
    1. The building blocks of CNNs
      1. Understanding CNNs and feature hierarchies
      2. Performing discrete convolutions
        1. Discrete convolutions in one dimension
        2. Padding inputs to control the size of the output feature maps
        3. Determining the size of the convolution output
        4. Performing a discrete convolution in 2D
      3. Subsampling layers
    2. Putting everything together – implementing a CNN
      1. Working with multiple input or color channels
      2. Regularizing an NN with dropout
      3. Loss functions for classification
    3. Implementing a deep CNN using TensorFlow
      1. The multilayer CNN architecture
      2. Loading and preprocessing the data
      3. Implementing a CNN using the TensorFlow Keras API
        1. Configuring CNN layers in Keras
        2. Constructing a CNN in Keras
    4. Gender classification from face images using a CNN
      1. Loading the CelebA dataset
      2. Image transformation and data augmentation
      3. Training a CNN gender classifier
    5. Summary
  17. Modeling Sequential Data Using Recurrent Neural Networks
    1. Introducing sequential data
      1. Modeling sequential data – order matters
      2. Representing sequences
      3. The different categories of sequence modeling
    2. RNNs for modeling sequences
      1. Understanding the RNN looping mechanism
      2. Computing activations in an RNN
      3. Hidden-recurrence versus output-recurrence
      4. The challenges of learning long-range interactions
      5. Long short-term memory cells
    3. Implementing RNNs for sequence modeling in TensorFlow
      1. Project one – predicting the sentiment of IMDb movie reviews
        1. Preparing the movie review data
        2. Embedding layers for sentence encoding
        3. Building an RNN model
        4. Building an RNN model for the sentiment analysis task
      2. Project two – character-level language modeling in TensorFlow
        1. Preprocessing the dataset
        2. Building a character-level RNN model
        3. Evaluation phase – generating new text passages
    4. Understanding language with the Transformer model
      1. Understanding the self-attention mechanism
        1. A basic version of self-attention
        2. Parameterizing the self-attention mechanism with query, key, and value weights
      2. Multi-head attention and the Transformer block
    5. Summary
  18. Generative Adversarial Networks for Synthesizing New Data
    1. Introducing generative adversarial networks
      1. Starting with autoencoders
      2. Generative models for synthesizing new data
      3. Generating new samples with GANs
      4. Understanding the loss functions of the generator and discriminator networks in a GAN model
    2. Implementing a GAN from scratch
      1. Training GAN models on Google Colab
      2. Implementing the generator and the discriminator networks
      3. Defining the training dataset
      4. Training the GAN model
    3. Improving the quality of synthesized images using a convolutional and Wasserstein GAN
      1. Transposed convolution
      2. Batch normalization
      3. Implementing the generator and discriminator
      4. Dissimilarity measures between two distributions
      5. Using EM distance in practice for GANs
      6. Gradient penalty
      7. Implementing WGAN-GP to train the DCGAN model
      8. Mode collapse
    4. Other GAN applications
    5. Summary
  19. Reinforcement Learning for Decision Making in Complex Environments
    1. Introduction – learning from experience
      1. Understanding reinforcement learning
      2. Defining the agent-environment interface of a reinforcement learning system
    2. The theoretical foundations of RL
      1. Markov decision processes
      2. The mathematical formulation of Markov decision processes
        1. Visualization of a Markov process
        2. Episodic versus continuing tasks
      3. RL terminology: return, policy, and value function
        1. The return
        2. Policy
        3. Value function
      4. Dynamic programming using the Bellman equation
    3. Reinforcement learning algorithms
      1. Dynamic programming
        1. Policy evaluation – predicting the value function with dynamic programming
        2. Improving the policy using the estimated value function
        3. Policy iteration
        4. Value iteration
      2. Reinforcement learning with Monte Carlo
        1. State-value function estimation using MC
        2. Action-value function estimation using MC
        3. Finding an optimal policy using MC control
        4. Policy improvement – computing the greedy policy from the action-value function
      3. Temporal difference learning
        1. TD prediction
        2. On-policy TD control (SARSA)
        3. Off-policy TD control (Q-learning)
    4. Implementing our first RL algorithm
      1. Introducing the OpenAI Gym toolkit
        1. Working with the existing environments in OpenAI Gym
        2. A grid world example
        3. Implementing the grid world environment in OpenAI Gym
      2. Solving the grid world problem with Q-learning
        1. Implementing the Q-learning algorithm
      3. A glance at deep Q-learning
        1. Training a DQN model according to the Q-learning algorithm
        2. Implementing a deep Q-learning algorithm
    5. Chapter and book summary
  20. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think
  21. Index

Product information

  • Title: Python Machine Learning - Third Edition
  • Author(s): Sebastian Raschka, Vahid Mirjalili
  • Release date: December 2019
  • Publisher(s): Packt Publishing
  • ISBN: 9781789955750