Open AccessMulti-parallax views synthesis for three-dimensional light-field display using unsupervised CNN
Author(s) -
Duo Chen,
Xinzhu Sang,
Peng Wang,
Xunbo Yu,
Hua Chun Wang
Publication year - 2018
Publication title -
optics express
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.394
H-Index - 271
ISSN - 1094-4087
DOI - 10.1364/oe.26.027585
Subject(s) - parallax , computer science , light field , artificial intelligence , convolutional neural network , view synthesis , computer vision , field (mathematics) , pixel , stereo display , position (finance) , artificial neural network , computer graphics (images) , rendering (computer graphics) , mathematics , finance , pure mathematics , economics
Multi-view applications have been used in a wide range, especially Three-Dimensional (3D) display. Since capturing dense multiple views for 3D light-field display is still a difficult work, view synthesis becomes an accessible way. Convolutional neural networks (CNN) has been used to synthesize new views of the scene. However, training targets are sometimes difficult to obtain, and it views are very difficult to synthesize at arbitrary positions. Here, an unsupervised network of Multi-Parallax View Net (MPVN) is proposed, which can synthesize multi-parallax views for 3D light-field display. Existing parallax views are re-projected to the target position to build input towers. The network is operated on these towers, and outputs a color tower and a selection tower. These two towers yield the final output image by per-pixel weight summing. MPVN adopts end-to-end unsupervised training to minimize prediction errors at existing positions. It can predict virtual views at any parallax position between existing views in a high quality. Experimental results demonstrate the validation of our proposed network, and SSIM of synthetic views are mostly over 0.95. We believe that this method can effectively provide enough views for 3D light-field display in the future work.