Open AccessCOLF‐GAN: Learning to axial super‐resolve focal stacks
Author(s) -
Xiao Zhaolin,
Liu Huan,
Jin Haiyan
Publication year - 2022
Publication title -
iet image processing
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.401
H-Index - 45
eISSN - 1751-9667
pISSN - 1751-9659
DOI - 10.1049/ipr2.12460
Subject(s) - stack (abstract data type) , weighting , computer science , generative adversarial network , focal length , generative grammar , cardinal point , adversarial system , field (mathematics) , process (computing) , algorithm , artificial intelligence , deep learning , optics , mathematics , lens (geology) , acoustics , physics , pure mathematics , programming language , operating system
In order to generate a denser focal stack, a cooperative generative adversarial network is proposed to learn the refocusing ability from light field imaging. To keep the axial continuity, the proposed framework is designed to learn features of a focal stack in both axial directions. Different from the classic generative adversarial network, our generative module consists of a forward prediction sub‐network and a backward prediction sub‐network, taking the forward‐and‐backward focal stacks as their inputs, respectively. The bi‐directional predictions are then fused by a weighting process, which is guided by an adversarial module. The proposed network is trained on light field focal stacks conducted via digital refocusing. Without loss of the refocus continuity, one can axial super‐resolve a focal stack by using the trained model. The effectiveness of the proposed algorithm on different types of focal stacks produced by both light fields and traditional camera shootings is validated. The experimental results indicate that the refocus variation of a focal stack can be well learned and predicted without a complete light field. Therefore, the proposed algorithm outperforms the traditional digital refocusing in terms of run‐time.