Premium
Deep Separation of Direct and Global Components from a Single Photograph under Structured Lighting
Author(s) -
Duan Z.,
Bieron J.,
Peers P.
Publication year - 2020
Publication title -
computer graphics forum
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.578
H-Index - 120
eISSN - 1467-8659
pISSN - 0167-7055
DOI - 10.1111/cgf.14159
Subject(s) - computer science , projector , encoder , deep learning , artificial intelligence , global illumination , specular reflection , separation (statistics) , structured light , high fidelity , binary number , computer vision , computer graphics (images) , optics , operating system , rendering (computer graphics) , physics , electrical engineering , machine learning , engineering , arithmetic , mathematics
We present a deep learning based solution for separating the direct and global light transport components from a single photograph captured under high frequency structured lighting with a co‐axial projector‐camera setup. We employ an architecture with one encoder and two decoders that shares information between the encoder and the decoders, as well as between both decoders to ensure a consistent decomposition between both light transport components. Furthermore, our deep learning separation approach does not require binary structured illumination, allowing us to utilize the full resolution capabilities of the projector. Consequently, our deep separation network is able to achieve high fidelity decompositions for lighting frequency sensitive features such as subsurface scattering and specular reflections. We evaluate and demonstrate our direct and global separation method on a wide variety of synthetic and captured scenes.