Augmenting industrial reality

Augmented reality (AR) is clearly poised to shift our world. Facebook just devoted a whole building, Building 8, to creating AR—they’re currently hiring camera systems architects and neural imaging engineers. Companies like Chevrolet are creating amazing examples of realtime AR—in which cars are digitally rendered so realistically you’d swear they were filmed (nope). The Economist is reporting on the space in a thoughtful and highly accurate way. The implication of all of this is that this is big business. And it is. Pokemon GO was a flash-trend that shone a spotlight on how this can excite people. It also proved that AR can be used to make money. AR is poised to make a big leap in the next five years.

And there’s only one space in which AR has deeply proven its utility in so far. Namely: manufacturing and industry—enterprise.

Two things became really clear while working on my recent book, Augmented Reality for the Industrial Enterprise. After digging into how AR is built, looking at missing content in the space, exploring how it is being applied in industry, and conducting 30 hours of interviews with people building AR systems, it became obvious that:

• Enterprise AR has found a foothold because of years of development—a long growth cycle—and improved optics/processing speed in computing.
• AR as a tool is useful. It is currently being used by big companies like Boeing, BMW, and Newport News shipbuilding (a $7-billion company). And it is saving time and money.

That’s where things are at now. There are also a handful of key developments in AR that point to where the future of AR is going. Two of them to pay close attention to right now: integration tools and vision systems.

Integrating AR into legacy systems

One of the key things missing in the space, as several people I interviewed pointed out, is the knowledge of how to integrate AR with business practices and literal mechanical systems within old companies. While the creators playing in AR are progressive, the world of traditional manufacturing is not. Some companies are 50-100 years old. Many traditional manufacturers operate a series of facilities. Often these facilities are one-offs, each of which is run differently from both an operating and a mechanical perspective. And all of this makes it challenging.

To seamlessly integrate AR with ideas and machines that have been used for decades, AR tools have to be created with an understanding of legacy systems. When building or hiring your enterprise AR team, you may think you’re looking for engineers and developers. But your first cut should be about whether they know manufacturing—through past experience or training.

As Dan Arczynski from Index AR says:

The reason why we are so effective is because our people are actually manufacturing people, construction people first—and engineers and designers and software developers. They’re AR people second. We train our people in our AR here, because we don’t want just software developers, we want those who can actually walk into a facility and talk directly to the folks running it and help them change things for the better.

Tech integration tools in manufacturing is a key area to watch. AR is a valuable tool in the IoT stack. And yet, few people are working in the AR integration space. While companies like Particle are doing a great job helping manufacturers get up to speed with sensors, data, and technology, they aren’t currently playing in AR. Other companies like Scope AR are experts in AR, but they aren’t playing in the other areas needed to upgrade legacy systems to “brilliant factory” level. Though they’re approaching the task differently, PTC with its ThingWorx Studio (part of the ThingWorx IIoT Platform) and Momentus are currently the only two companies, from my research so far, working across both of those worlds. The bridge between manufacturing, and IoT (including AR) is a crucial one. More bridge builders are needed.

Combining computer vision and AR

The second key area to pay attention to is the effect of computer vision on AR. Computer vision has advanced significantly in the past three years, making things possible now that simply weren’t before. Significant improvements can be seen in:

• Processing power.
• Depth sensing.
• Multi-camera phones/devices.
• Broader use of simultaneous localization and mapping or SLAM (a technique used in robotics for mapping unknown environments, among other things).

All of these point to a future of AR where traditional ways of doing things soon will not apply.

Because computer vision uses visual features of the landscape–the things already in the world as you look around–to register the camera, it doesn’t have the limitations that sensors or markers do. Markers still require engineering and application, they work only from a specified distance, and they are clunky from a UX standpoint. Who wants to precisely apply and calibrate or “register” markers for every machine or every factory? Computer vision, on the other hand, can process frames of photos in real time—making tracking easy. In his paper on “Computer Vision for Augmented Reality,” professor Vincent Lepetit, formerly of the the Ecole Polytechnique Federale de Lausanne (EPFL) Computer Vision Laboratory, calls this “tracking by detection.” It is an old study, and one that’s still highly relevant and on point.

More recently, in January, Apple patented an augmented reality device that works using object recognition. Hackers are also experimenting with computer vision as a way to create AR experiences using object recognition and common web technologies. Even big companies like AT&T recognize the benefits of markerless AR—without speaking to the how behind it, at least they seem to see where things are going.

Object recognition tools are getting better and depth-sensing cameras are beginning to make markers unnecessary, even in industry. What will replace them: augmented reality triggered by the world around us. We’ll have cameras that can recognize whole landscapes and process changes in real time. And perhaps invisible, light-driven indicators that act as markers (check out this under-the-radar HP patent from 2016). This is where the space is going, and these are the areas you should build toward.

AR is advancing. What will continue to move it forward, at least in the near-term, is enterprise adoption. Big companies. Big industries. Big budgets.

To learn more about the future of AR in the industrial enterprise, check out Augmented Reality for the Industrial Enterprise, where I speak with the heads of AR programs at Caterpillar and Boeing, among others, about how and why they have already adopted AR–and what they’re getting out of it.

Because the benefit their work, ultimately, will come to all of us.

---

This post is a collaboration between O’Reilly and PTC. See our statement of editorial independence.