Video description
Much progress has been made over the past decade on process and tooling for managing large-scale, multitier cloud apps and APIs, but there is far less common knowledge on best practices for managing machine-learned models (classifiers, forecasters, etc.), especially beyond the modeling, optimization, and deployment process once these models are in production.
A key mindset shift required to address these issues is understanding that model development is different than software development in fundamental ways. David Talby (Pacific AI) shares real-world case studies showing why this is true and explains what you can do about it, covering key best practices that executives, solution architects, and delivery teams must take into account when committing to successfully deliver and operate data science-intensive systems in the real world.
Topics include:
- Concept drift (Machine-learned models begin degrading as soon as they’re deployed and must adapt to a changing environment.)
- Locality and limited reuse and generalization of models
- A/B testing challenges, which make it very hard in practice to know which model will perform better in production
- Semisupervised and adversarial learning scenarios, which require modeling and optimizing models only once they’re in production
- The impact of all of the above on product planning, staffing, and client expectation management
This session was recorded at the 2019 O'Reilly Strata Data Conference in San Francisco.
Product information
- Title: Executive Briefing: Why machine-learned models crash and burn in production and what to do about it
- Author(s):
- Release date: October 2019
- Publisher(s): O'Reilly Media, Inc.
- ISBN: 0636920340096
You might also like
article
Have ChatGPT Ask You Questions
ChatGPT Shortcuts shows future prompt engineers how to harness the full potential of the state-of-the-art AI …
video
GenAI Essentials for Everyone - Overview
Our team of experts has hand-selected and organized the most crucial concepts and practical applications of …
video
Spotlight on Data: Machine Learning in Production at Google Scale with Todd Underwood
You can propel your business forward with AI-centric approaches to solving customer needs, but to be …
article
Use GitHub Copilot: Additional Tips
Using GitHub Copilot can feel like magic. The tool automatically fills out entire blocks of code--but …