3D Visual Communications

5.9 Free-View Point Video

A free viewpoint 3D video system allows end users to arbitrarily change the viewing position and angle to enrich viewers' immersive experiences. The key bridging technology between content generation, transmission, and display stages of a 3D video system is the 3D scene representation. As discussed in Chapter 2, there are two fundamental approaches to defining 3D scenes: image-based and geometry-based representation [59].

Image-based representation uses a set of camera views arranged in a 1D/2D array to represent the 3D world, which can be represented as a set of sparse sampling of the plenoptic function [60]. There have been several similar requirements and operations in video processing applications, such as spatial upsampling or frame rate upconversion, where new pixels are generated by utilizing the information from the existing pixels. For the free viewpoint 3D application, a new view can be synthesized from this sparse set of samples of the plenoptic function. The main advantage is the high quality of virtual view synthesis and an avoidance of 3D scene reconstruction. The computation complexity via image-based representation is proportional to the number of pixels in the reference and output images, but in general not to the geometric complexity such as triangle counts. However, the synthesis ability provided by image-based representation is limited by the viewing angle covered by all the existing views. Besides, the quality of the synthesized views ...