KinFu MOT: KinectFusion with Moving Objects Tracking

Using a depth camera, the KinectFusion algorithm permits tracking the camera poses and building a dense 3D reconstruction of the environment simultaneously in real-time. We present an extension to this algorithm that allows additionally the concurrent tracking and reconstruction of several moving objects within the perceived environment. This is achieved through an expansion of the GPU processing pipeline by several new functionalities. Our system detects moving objects from the registration results and it creates a separate storing volume for such objects. Each object and the background are tracked and reconstructed individually. Since the size of an object is uncertain at the moment of detection the storing volume grows dynamically. Moreover, a sliding reduction method stabilizes the tracking of objects with ambiguous registrations. We provide experimental results showing the effects of our modified matching strategy. Furthermore, we demonstrate the system’s ability to deal with three different challenging situations containing a moving robot.