Reliable Machine Learning

Chapter 3. Basic Introduction to Models

Most of this book is about managing ML systems and production-level ML pipelines. This involves work that is quite different from the work often performed by many data scientists and ML researchers, who try to spend their days developing new predictive models and methods that can squeeze out another percentage point of accuracy. Instead, in this book, we focus on ensuring that a system that includes an ML model exhibits consistent, robust, and reliable system-level behavior. In some ways, this system-level behavior is independent of the actual model type, how good the model is, or other solely model-related considerations. Still, in certain key situations, it is not independent of these considerations. Our goal in this chapter is to give you enough background to understand which situation you are in when the alarms start to go off or the pagers start to fire for your production system.

We will say at the outset that our goal here is not to teach you everything about how to build ML models, which models might be good for what problems, or how to become a data scientist. That would be a book (or more) all to itself, and many excellent texts and online courses cover these aspects.

Instead of going too deep into the minutia, in this chapter our goal is to give a quick reminder about what ML models are and how they work. We’ll also provide some key questions that machine learning operations (MLOps) folks should ask about the models in their system ...