An approach to stereo-point cloud registration using image homographies

A mobile robot equipped with a stereo camera can measure both the video image of a scene and the visual disparity in the scene. The disparity image can be used to generate a collection of points, each representing the location of a surface in the visual scene as a 3D point with respect to the location of the stereo camera: a point cloud. If the stereo camera is moving, e.g., mounted on a moving robot, aligning these scans becomes a difficult, and computationally expensive problem. Many finely tuned versions of the iterative closest point algorithm (ICP) have been used throughout robotics for registration of these sets of scans. However, ICP relies on theoretical convergence to the nearest local minimum of the dynamical system: there is no guarantee that ICP will accurately align the scans. In order to address two problems with ICP, convergence time and accuracy of convergence, we have developed an improvement by using salient keypoints from successive video images to calculate an affine transformation estimate of the camera location. This transformation, when applied to the target point cloud, provides ICP an initial guess to reduce the computational time required for point cloud registration and improve the quality of registration. We report ICP convergence times with and without image information for a set of stereo data point clouds to demonstrate the effectiveness of the approach.