Robust Global Motion Compensation in Presence of Predominant Foreground

Global motion compensation (GMC) removes intentional and unwanted camera motion. GMC is widely applicable for video stitching and, as a pre-processing module, for motion-based video analysis. While state-of-the-art GMC algorithms generally estimate homography satisfactorily between consecutive frames, their performances deteriorate on real-world unconstrained videos, for instance, videos with predominant foreground, e.g., moving objects or human, or uniform background. Since GMC transformation of frames to the global motion-compensated coordinate is done by cascading homographies, failure in GMC of a single frame drastically harms the final result. Thus, we propose a robust GMC, termed RGMC, based on homography estimation using keypoint matches. RGMC first suppresses the foreground impact by clustering the keypoint matches and removing those pertaining to the foreground, as well as erroneous matches. For homography verification, we propose a probabilistic model that combines keypoint matching error, consistency of edges after homograhy transformation, the motion history, and prior camera motion information. Experimental results on the Sports Videos in the Wild, Holleywood2, and HMDB51 datasets demonstrate the superiority of RGMC.