Premium
Video frame interpolation via optical flow estimation with image inpainting
Author(s) -
Liu Xiaozhang,
Liu Hui,
Lin Yuxiu
Publication year - 2020
Publication title -
international journal of intelligent systems
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.291
H-Index - 87
eISSN - 1098-111X
pISSN - 0884-8173
DOI - 10.1002/int.22285
Subject(s) - motion interpolation , optical flow , computer vision , computer science , artificial intelligence , interpolation (computer graphics) , frame (networking) , image scaling , motion compensation , residual frame , frame rate , motion estimation , inpainting , block matching algorithm , inter frame , reference frame , video tracking , image processing , image (mathematics) , video processing , telecommunications
As we all know, video frame rate determines the quality of the video. The higher the frame rate, the smoother the movements in the picture, the clearer the information expressed, and the better the viewing experience for people. Video interpolation aims to increase the video frame rate by generating a new frame image using the relevant information between two consecutive frames, which is essential in the field of computer vision. The traditional motion compensation interpolation method will cause holes and overlaps in the reconstructed frame, and is easily affected by the quality of optical flow. Therefore, this paper proposes a video frame interpolation method via optical flow estimation with image inpainting. First, the optical flow between the input frames is estimated via combined local and global‐total variation (CLG‐TV) optical flow estimation model. Then, the intermediate frames are synthesized under the guidance of the optical flow. Finally, the nonlocal self‐similarity between the video frames is used to solve the optimization problem, to fix the pixel loss area in the interpolated frame. Quantitative and qualitative experimental results show that this method can effectively improve the quality of optical flow estimation, generate realistic and smooth video frames, and effectively increase the video frame rate.