Open AccessVirtual view synthesis for 3D light-field display based on scene tower blending
Author(s) -
Duo Chen,
Xinzhu Sang,
Peng Wang,
Xunbo Yu,
Xin Gao,
Binbin Yan,
Huachun Wang,
Shuai Qi,
Xiaoqian Ye
Publication year - 2021
Publication title -
optics express
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.394
H-Index - 271
ISSN - 1094-4087
DOI - 10.1364/oe.419069
Subject(s) - view synthesis , computer science , light field , parallax , artificial intelligence , computer vision , tower , bottleneck , viewpoints , computer graphics (images) , field (mathematics) , stereo display , depth of field , field of view , virtual reality , rendering (computer graphics) , mathematics , art , civil engineering , pure mathematics , engineering , visual arts , embedded system
Three-dimensional (3D) light-field display has achieved a great improvement. However, the collection of dense viewpoints in the real 3D scene is still a bottleneck. Virtual views can be generated by unsupervised networks, but the quality of different views is inconsistent because networks are separately trained on each posed view. Here, a virtual view synthesis method for the 3D light-field display based on scene tower blending is presented, which can synthesize high quality virtual views with correct occlusions by blending all tower results, and dense viewpoints on 3D light-field display can be provided with smooth motion parallax. Posed views are combinatorially input into diverse unsupervised CNNs to predict respective input-view towers, and towers of the same viewpoint are fused together. All posed-view towers are blended as a scene color tower and a scene selection tower, so that 3D scene distributions at different depth planes can be accurately estimated. Blended scene towers are soft-projected to synthesize virtual views with correct occlusions. A denoising network is used to improve the image quality of final synthetic views. Experimental results demonstrate the validity of the proposed method, which shows outstanding performances under various disparities. PSNR of the virtual views are about 30 dB and SSIM is above 0.91. We believe that our view synthesis method will be helpful for future applications of the 3D light-field display.