Machine learning needs machine teaching
The O’Reilly Data Show Podcast: Mark Hammond on applications of reinforcement learning to manufacturing and industrial automation.
The Brazos Wind Farm (source: Leaflet on Wikimedia Commons)
In this episode of the Data Show, I spoke with Mark Hammond, founder and CEO of Bonsai, a startup at the forefront of developing AI systems in industrial settings. While many articles have been written about developments in computer vision, speech recognition, and autonomous vehicles, I’m particularly excited about near-term applications of AI to manufacturing, robotics, and industrial automation. In a recent post, I outlined practical applications of reinforcement learning (RL)—a type of machine learning now being used in AI systems. In particular, I described how companies like Bonsai are applying RL to manufacturing and industrial automation. As researchers explore new approaches for solving RL problems, I expect many of the first applications to be in industrial automation.
Here are some highlights from our conversation:
Learn faster. Dig deeper. See farther.
Machine learning and machine teaching
Everyone is so focused on making better and faster learning algorithms; what do we do when we have it? Let’s just suppose that you now have an algorithm that can learn as well as or better than humans. How do we use that, how do we apply that in a predictable, scalable, repeatable way toward the objectives that we want to apply it toward?
… I thought about that for a while, and it’s one of those things where the answer is obvious in hindsight, but until you sit down and really chew on it, it doesn’t jump out at you. And it’s that, by design, if you’re building a learning system—if you want to program it—you have to teach it. Machine teaching and machine learning are necessary complements to one another; you need both. And for the large part, most of what comprises machine teaching these days consists of giant label data sets.
… You need machine teaching and machine learning. It dawned on me that this was the core abstraction that was going to make it possible for us to start applying all of this stuff more broadly across all the myriad use cases that we see in the real world without having to turn all of the people who are looking to use it into experts in machine learning and data science. It’s what enabled me to realize what Bonsai’s mission is: to enable your subject matter experts (a chemical engineer or a mechanical engineer, someone who is very, very well versed in whatever their domain is but not necessarily in machine learning or data science) to take that expertise and use it as the foundation for describing what to teach and then automating the underlying pieces for how you can actually effectively learn that.
Common use cases for reinforcement learning
In a lot of cases, the system you are working with is undergoing some form of tuning. We see a lot of this. It might be that you have an HVAC system and you are looking to optimize across the lifetime of the equipment, the comfort of the user, and the energy consumption of the device, which is its own complex optimization problem. You do this by virtue of tuning a bunch of parameters or controlling a bunch of different capabilities on the device.
… So, in many cases, a lot of what our users are applying the system toward is not necessarily strict real-time control. Certainly, we do that as well, and we put out a paper recently showing dexterous robotic control and manipulation using these kinds of techniques. You can absolutely do that, and we have customers who do that. A very common thing that we see across a lot of the real-world applications is more a tuning characteristic.
Related resources:
- “Practical applications of reinforcement learning in industry”: an overview of commercial and industrial applications of reinforcement learning
- “Introducing RLlib – A composable and scalable reinforcement learning library”: this new software makes the task of training RL models much more accessible
- Deep reinforcement learning in the enterprise—Bridging the gap from games to industry (2017 Artificial Intelligence Conference presentation by Mark Hammond)
- Ray: A distributed execution framework for reinforcement learning applications (2017 Artificial Intelligence Conference presentation by Ion Stoica)
- Deep reinforcement learning for robotics (2016 Artificial Intelligence Conference presentation by Pieter Abbeel)
- “Neuroevolution: A different kind of deep learning”