Multi-Scheme Frame Rate Up-Conversion Using Space-Time Saliency

We propose a novel frame rate up-conversion algorithm, which adaptively selects multiple motion estimation (ME) schemes to generate absent frames according to the space-time saliency. We first use a space-time saliency classifier to determine which frames are most important to a human observer. Based on the varying saliency over time, we adopt high-cost ME scheme on high-salient frame but low-cost ME scheme on low-salient frame. Automatic switch of multiple ME schemes takes human visual attention into consideration, so that unnecessary computations cannot be invested into the frames of no interest. Experimental results show that our algorithm provides a better visual interpolation quality, while removing some redundant computations.