What does an augmented reality system look like?

Let's consider the following figure:

As we can see here, the camera captures real-world video to get the reference point. The graphics system generates the virtual objects that need to be overlaid on top of the video. Now, the video-merging block is where all the magic happens. This block should be smart enough to understand how to overlay the virtual objects on top of the real world in the best way possible.

Get OpenCV 3.x with Python By Example - Second Edition now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.