Robust Multithreaded Object Tracker through Occlusions for Spatial Augmented Reality

A spatial augmented reality ( SAR ) system enables a virtual image to be projected onto the surface of a real‐world object and the user to intuitively control the image using a tangible interface. However, occlusions frequently occur, such as a sudden change in the lighting environment or the generation of obstacles. We propose a robust object tracker based on a multithreaded system, which can track an object robustly through occlusions. Our multithreaded tracker is divided into two threads: the detection thread detects distinctive features in a frame‐to‐frame manner, and the tracking thread tracks features periodically using an optical‐flow‐based tracking method. Consequently, although the speed of the detection thread is considerably slow, we achieve real‐time performance owing to the multithreaded configuration. Moreover, the proposed outlier filtering automatically updates a random sample consensus distance threshold for eliminating outliers according to environmental changes. Experimental results show that our approach tracks an object robustly in real‐time in an SAR environment where there are frequent occlusions occurring from augmented projection images.