Machine Learning and the Production Gap
Mechanical eye illustration, (source: Sirylok on Dreamstime.com)
The biggest problem facing machine learning today isn’t the need for better algorithms; it isn’t the need for more computing power to train models; it isn’t even the need for more skilled practitioners. It’s getting machine learning from the researcher’s laptop to production. That’s the real gap. It’s one thing to build a model; it’s another thing altogether to embody that model in an application and deploy it successfully in production.
That’s where Emmanuel Ameisen’s book, Building Machine Learning Applications, comes in. When I first met Emmanuel, three or four years ago, what impressed me wasn’t his expertise in building models—though he clearly had that. (I first learned about Emmanuel through articles on his blog.) He clearly cared about the whole process: not just developing algorithms, finding and cleaning data, and training models, but building a working application and putting it in production.
Learn faster. Dig deeper. See farther.
That’s what his new book is about. The development process doesn’t end with a model. It ends with that model that’s deployed. You can’t just talk about programming or training; you’ve got to make this work in the real world.
Emmanuel starts at the beginning: what are the goals for the product, and how do you refine those goals into something that can be reasonably implemented? You need to understand whether a problem can be solved—and if not, how to reframe the problem so that it can be. You need to define metrics that show how your system is performing, and whether you’re making progress. You need to collect relevant data for training, and deploy pipelines that will feed data to the model when it is in production. Making a product that works in the real world also includes understanding how to deploy the model; monitoring performance after deployment; and ongoing maintenance and updates.
Maintenance may be the most important issue. In the last few years, operations teams have learned a lot about continuous deployment and delivery (CI/CD). The question facing us now is how machine learning applications fit into this model. How do you monitor ML applications, and what kind of monitoring is needed? How do you detect model drift? These concepts are new to the ongoing conversation about monitoring and observability. How do you practice rapid deployment when training a model can take hours or days?
There are many books out there that talk about machine learning. But this is the only one I know of that covers the entire process, end-to-end, in approachable and practical terms. It’s the only one that focuses on the biggest machine learning problem of all: getting the model off of your laptop and into production.