High-Resolution Image Synthesis from Video Sequence by Light Field

We propose a novel method to synthesize high-resolution images from image sequences taken with a moving video camera. Each frame in the image sequence is a part of the photographed object. Our method integrates these frames to generate high-resolution images of object by constructing a light field, which is quite different from general mosaic methods. In light fields constructed straightforwardly, blur and discontinuity are introduced into synthesized images by depth variation of the object. In our method, the light field is optimized to remove blur and discontinuity so clear images can be synthesized. We find the optimum light field for generating sharp unblurred images by reparameterizing light field and evaluating sharpness of synthesized images from each light field. The optimized light field is adapted to the depth variation of the object surface, but the exact shape of the object is not necessary. High resolution images that are impractical in the real system can be virtually synthesized from the light field. Results of the experiment applied to a book surface demonstrate the effectiveness of the proposed method.