Chapter 9. MLOps and Continuous Delivery for ML (CD4ML)
That anxiety makes its appearance is the pivot upon which everything turns.
Søren Kierkegaard, The Concept of Anxiety
It’s 10:36 a.m. Dana is pairing with Ted, an infrastructure engineer, to deploy the new model her team has been working on for several months. The energy in the room is mixed with determination and anxiety—it’s a new model for a high-profile release. They’ve been testing the model for three weeks, but the next hurdle—deployment to production—has typically been fraught with issues and numerous retries.
As they navigate the labyrinthine web of deployment scripts, configuration files, and infrastructure components, Dana couldn’t help but feel that something was amiss. She wasn’t confident that their test dataset was representative of what the model would be seeing in production. It didn’t help that the complexity of the system was so overwhelming—the sheer number of moving parts made it difficult to get a sense of where things might go wrong.
It’s 12:45 p.m. Dana and Ted completed the last of their deployment procedures 10 minutes ago, but the trickling stream of alerts is a cruel reminder that something has gone awry.
It’s 7:10 p.m. After hours of troubleshooting, a fix has finally been deployed. Dana and Ted exhale relief and finally go home, exhausted.
A week has passed, and one Tuesday morning Dana hears a scrape-triple-knock on her Slack. A message at 8:45 a.m.—it is Sarah from product analytics. She informs ...
Get Effective Machine Learning Teams now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
