Hands-On Machine Learning with C++

Hands-On Machine Learning with C++

by Kirill Kolodiazhnyi
Released May 2020
Publisher(s): Packt Publishing
ISBN: 9781789955330

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Implement supervised and unsupervised machine learning algorithms using C++ libraries such as PyTorch C++ API, Caffe2, Shogun, Shark-ML, mlpack, and dlib with the help of real-world examples and datasets

Key Features

  • Become familiar with data processing, performance measuring, and model selection using various C++ libraries
  • Implement practical machine learning and deep learning techniques to build smart models
  • Deploy machine learning models to work on mobile and embedded devices

Book Description

C++ can make your machine learning models run faster and more efficiently. This handy guide will help you learn the fundamentals of machine learning (ML), showing you how to use C++ libraries to get the most out of your data. This book makes machine learning with C++ for beginners easy with its example-based approach, demonstrating how to implement supervised and unsupervised ML algorithms through real-world examples.

This book will get you hands-on with tuning and optimizing a model for different use cases, assisting you with model selection and the measurement of performance. You'll cover techniques such as product recommendations, ensemble learning, and anomaly detection using modern C++ libraries such as PyTorch C++ API, Caffe2, Shogun, Shark-ML, mlpack, and dlib. Next, you'll explore neural networks and deep learning using examples such as image classification and sentiment analysis, which will help you solve various problems. Later, you'll learn how to handle production and deployment challenges on mobile and cloud platforms, before discovering how to export and import models using the ONNX format.

By the end of this C++ book, you will have real-world machine learning and C++ knowledge, as well as the skills to use C++ to build powerful ML systems.

What you will learn

  • Explore how to load and preprocess various data types to suitable C++ data structures
  • Employ key machine learning algorithms with various C++ libraries
  • Understand the grid-search approach to find the best parameters for a machine learning model
  • Implement an algorithm for filtering anomalies in user data using Gaussian distribution
  • Improve collaborative filtering to deal with dynamic user preferences
  • Use C++ libraries and APIs to manage model structures and parameters
  • Implement a C++ program to solve image classification tasks with LeNet architecture

Who this book is for

You will find this C++ machine learning book useful if you want to get started with machine learning algorithms and techniques using the popular C++ language. As well as being a useful first course in machine learning with C++, this book will also appeal to data analysts, data scientists, and machine learning developers who are looking to implement different machine learning models in production using varied datasets and examples. Working knowledge of the C++ programming language is mandatory to get started with this book.

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. Hands-On Machine Learning with C++
  3. About Packt
    1. Why subscribe?
  4. Contributors
    1. About the author
    2. About the reviewers
    3. Packt is searching for authors like you
  5. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Download the color images
      3. Conventions used
    4. Get in touch
      1. Reviews
  6. Section 1: Overview of Machine Learning
  7. Introduction to Machine Learning with C++
    1. Understanding the fundamentals of ML
      1. Venturing into the techniques of ML
        1. Supervised learning
        2. Unsupervised learning
      2. Dealing with ML models
      3. Model parameter estimation
    2. An overview of linear algebra
      1. Learning the concepts of linear algebra
      2. Basic linear algebra operations
      3. Tensor representation in computing
      4. Linear algebra API samples
        1. Using Eigen
        2. Using xtensor
        3. Using Shark-ML
        4. Using Dlib
    3. An overview of linear regression
      1. Solving linear regression tasks with different libraries
        1. Solving linear regression tasks with Eigen
        2. Solving linear regression tasks with Shogun
        3. Solving linear regression tasks with Shark-ML
        4. Linear regression with Dlib
    4. Summary
    5. Further reading
  8. Data Processing
    1. Technical requirements
    2. Parsing data formats to C++ data structures
      1. Reading CSV files with the Fast-CPP-CSV-Parser library
      2. Preprocessing CSV files
      3. Reading CSV files with the Shark-ML library
      4. Reading CSV files with the Shogun library
      5. Reading CSV files with the Dlib library
      6. Reading JSON files with the RapidJSON library
      7. Writing and reading HDF5 files with the HighFive library
    3. Initializing matrix and tensor objects from C++ data structures
      1. Eigen
      2. Shark-ML
      3. Dlib
      4. Shogun
    4. Manipulating images with the OpenCV and Dlib libraries
      1. Using OpenCV
      2. Using Dlib
    5. Transforming images into matrix or tensor objects of various libraries
      1. Deinterleaving in OpenCV
      2. Deinterleaving in Dlib
    6. Normalizing data
      1. Normalizing with Eigen
      2. Normalizing with Shogun
      3. Normalizing with Dlib
      4. Normalizing with Shark-ML
    7. Summary
    8. Further reading
  9. Measuring Performance and Selecting Models
    1. Technical requirements
    2. Performance metrics for ML models
      1. Regression metrics
        1. Mean squared error and root mean squared error
        2. Mean absolute error
        3. R squared
        4. Adjusted R squared
      2. Classification metrics
        1. Accuracy
        2. Precision and recall
        3. F-score
        4. AUC–ROC
        5. Log-Loss
    3. Understanding the bias and variance characteristics
      1. Bias
      2. Variance
      3. Normal training
      4. Regularization
        1. L1 regularization – Lasso
        2. L2 regularization – Ridge
        3. Data augmentation
        4. Early stopping
        5. Regularization for neural networks
    4. Model selection with the grid search technique
      1. Cross-validation
        1. K-fold cross-validation
      2. Grid search
      3. Shogun example
      4. Shark-ML example
      5. Dlib example
    5. Summary
    6. Further reading
  10. Section 2: Machine Learning Algorithms
  11. Clustering
    1. Technical requirements
    2. Measuring distance in clustering
      1. Euclidean distance
      2. Squared Euclidean distance
      3. Manhattan distance
      4. Chebyshev distance
    3. Types of clustering algorithms
      1. Partition-based clustering algorithms
        1. Distance-based clustering algorithms
        2. Graph theory-based clustering algorithms
      2. Spectral clustering algorithms
      3. Hierarchical clustering algorithms
      4. Density-based clustering algorithms
      5. Model-based clustering algorithms
    4. Examples of using the Shogun library for dealing with the clustering task samples
      1. GMM with Shogun
      2. K-means clustering with Shogun
      3. Hierarchical clustering with Shogun
    5. Examples of using the Shark-ML library for dealing with the clustering task samples
      1. Hierarchical clustering with Shark-ML
      2. K-means clustering with Shark-ML
    6. Examples of using the Dlib library for dealing with the clustering task samples
      1. K-means clustering with Dlib
      2. Spectral clustering with Dlib
      3. Hierarchical clustering with Dlib
      4. Newman modularity-based graph clustering algorithm with Dlib
      5. Chinese Whispers – graph clustering algorithm with Dlib
    7. Plotting data with C++
    8. Summary
    9. Further reading
  12. Anomaly Detection
    1. Technical requirements
    2. Exploring the applications of anomaly detection
    3. Learning approaches for anomaly detection
      1. Detecting anomalies with statistical tests
      2. Detecting anomalies with the Local Outlier Factor method
      3. Detecting anomalies with isolation forest
      4. Detecting anomalies with One-Class SVM (OCSVM)
      5. Density estimation approach (multivariate Gaussian distribution) for anomaly detection
    4. Examples of using different C++ libraries for anomaly detection
      1. C++ implementation of the isolation forest algorithm for anomaly detection
      2. Using the Dlib library for anomaly detection
        1. One-Cass SVM with Dlib
        2. Multivariate Gaussian model with Dlib
      3. OCSVM with Shogun
      4. OCSVM with Shark-ML
    5. Summary
    6. Further reading
  13. Dimensionality Reduction
    1. Technical requirements
    2. An overview of dimension reduction methods
      1. Feature selection methods
      2. Dimensionality reduction methods
    3. Exploring linear methods for dimension reduction
      1. Principal component analysis
      2. Singular value decomposition
      3. Independent component analysis
      4. Linear discriminant analysis
      5. Factor analysis
      6. Multidimensional scaling
    4. Exploring non-linear methods for dimension reduction
      1. Kernel PCA
      2. IsoMap
      3. Sammon mapping
      4. Distributed stochastic neighbor embedding
      5. Autoencoders
    5. Understanding dimension reduction algorithms with various С++ libraries
      1. Using the Dlib library
        1. PCA
          1. Data compression with PCA
        2. LDA
        3. Sammon mapping
      2. Using the Shogun library
        1. PCA
        2. Kernel PCA
        3. MDS
        4. IsoMap
        5. ICA
        6. Factor analysis
        7. t-SNE
      3. Using the Shark-ML library
        1. PCA
        2. LDA
    6. Summary
    7. Further reading
  14. Classification
    1. Technical requirements
    2. An overview of classification methods
    3. Exploring various classification methods
      1. Logistic regression
      2. KRR
      3. SVM
      4. kNN method
      5. Multi-class classification
    4. Examples of using C++ libraries for dealing with the classification task
      1. Using the Shogun library
        1. With logistic regression
        2. With SVMs
        3. With the kNN algorithm
      2. Using the Dlib library
        1. With KRR
        2. With SVM
      3. Using the Shark-ML library
        1. With logistic regression
        2. With SVM
        3. With the kNN algorithm
    5. Summary
    6. Further reading
  15. Recommender Systems
    1. Technical requirements
    2. An overview of recommender system algorithms
      1. Non-personalized recommendations
      2. Content-based recommendations
      3. User-based collaborative filtering
      4. Item-based collaborative filtering
      5. Factorization algorithms
      6. Similarity or preferences correlation
        1. Pearson's correlation coefficient
        2. Spearman's correlation
        3. Cosine distance
      7. Data scaling and standardization
      8. Cold start problem
      9. Relevance of recommendations
      10. Assessing system quality
    3. Understanding collaborative filtering method details
    4. Examples of item-based collaborative filtering with C++
      1. Using the Eigen library
      2. Using the mlpack library
    5. Summary
    6. Further reading
  16. Ensemble Learning
    1. Technical requirements
    2. An overview of ensemble learning
      1. Using a bagging approach for creating ensembles
      2. Using a gradient boosting method for creating ensembles
      3. Using a stacking approach for creating ensembles
      4. Using the random forest method for creating ensembles
        1. Decision tree algorithm overview
        2. Random forest method overview
    3. Examples of using C++ libraries for creating ensembles
      1. Ensembles with Shogun
        1. Using gradient boosting with Shogun
        2. Using random forest with Shogun
      2. Ensembles with Shark-ML
        1. Using random forest with Shark-ML
        2. Using a stacking ensemble with Shark-ML
    4. Summary
    5. Further reading
  17. Section 3: Advanced Examples
  18. Neural Networks for Image Classification
    1. Technical requirements
    2. An overview of neural networks
      1. Neurons
      2. The perceptron and neural networks
      3. Training with the backpropagation method
        1. Backpropagation method modes
          1. Stochastic mode
          2. Batch mode
          3. Mini-batch mode
        2. Backpropagation method problems
        3. The backpropagation method – an example
      4. Loss functions
      5. Activation functions
        1. The stepwise activation function
        2. The linear activation function
        3. The sigmoid activation function
        4. The hyperbolic tangent
        5. Activation function properties
      6. Regularization in neural networks
        1. Different methods for regularization
      7. Neural network initialization
        1. Xavier initialization method
        2. He initialization method
    3. Delving into convolutional networks
      1. Convolution operator
      2. Pooling operation
      3. Receptive field
      4. Convolution network architecture
    4. What is deep learning?
    5. Examples of using C++ libraries to create neural networks
      1. Simple network example for the regression task
        1. Dlib
        2. Shogun
        3. Shark-ML
          1. Architecture definition
          2. Loss function definition
          3. Network initialization
          4. Optimizer configuration
          5. Network training
          6. The complete programming sample
    6. Understanding image classification using the LeNet architecture
      1. Reading the training dataset
        1. Reading dataset files
        2. Reading the image file
      2. Neural network definition
      3. Network training
    7. Summary
    8. Further reading
  19. Sentiment Analysis with Recurrent Neural Networks
    1. Technical requirements
    2. An overview of the RNN concept
    3. Training RNNs using the concept of backpropagation through time
    4. Exploring RNN architectures
      1. LSTM
      2. GRUs
      3. Bidirectional RNN
      4. Multilayer RNN
    5. Understanding natural language processing with RNNs
      1. Word2Vec
      2. GloVe
    6. Sentiment analysis example with an RNN
    7. Summary
    8. Further reading
  20. Section 4: Production and Deployment Challenges
  21. Exporting and Importing Models
    1. Technical requirements
    2. ML model serialization APIs in C++ libraries
      1. Model serialization with Dlib
      2. Model serialization with Shogun
      3. Model serialization with Shark-ML
      4. Model serialization with PyTorch
        1. Neural network initialization
        2. Using the torch::save and torch::load functions
        3. Using PyTorch archive objects
    3. Delving into ONNX format
      1. Loading images into Caffe2 tensors
      2. Reading the class definition file
    4. Summary
    5. Further reading
  22. Deploying Models on Mobile and Cloud Platforms
    1. Technical requirements
    2. Image classification on Android mobile
      1. The mobile version of the PyTorch framework
      2. Using TorchScript for a model snapshot
      3. The Android Studio project
        1. The UI and Java part of the project
        2. The C++ native part of the project
    3. Machine learning in the cloud – using Google Compute Engine
      1. The server
      2. The client
      3. Service deployment
    4. Summary
    5. Further reading
  23. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Hands-On Machine Learning with C++
  • Author(s): Kirill Kolodiazhnyi
  • Release date: May 2020
  • Publisher(s): Packt Publishing
  • ISBN: 9781789955330