Robust Scene Stitching in Large Scale Mobile Mapping

This paper presents a solution for the loop closure problem in an image-based mobile mapping context. A van equipped with stereo cameras collects recordings in an urban environment, simultaneously monitoring GPS information. Using Structure-from-Motion, the position of the van and the surroundings are retrieved. The determination of the translation and orientation of the van’s position is recursive: a slight drift can gradually build up to flawed localizations. One can rely on the GPS information to perform adjustments, but its accuracy is not adequate to yield a model with high precision. Yet, visual loop-closing – recognizing that a location is revisited – may help mitigate the issue. The current system does not take into account possible reoccurrences of identical features in distant recordings. This paper adds such loop closure. Local feature matching in two stages detects when a particular site is revisited, in order to enforce correspondences between images, that may have been taken with large time lapses in between. Our system relies on GPS but does not use odometric information. We extend the original image-to-image matching approach to a pose-to-pose matching approach, combining several images and achieving robust scene matching results. Parameter optimization is followed by extensive experiments. Our pipeline, which facilitates parallel execution, reaches matching rates higher than those reported for typical state-of-the-art algorithms. We also demonstrate robustness to odometric inconsistencies resulting from poor prior model build-up. Loop closure is crucial for high-accuracy models. The current stateof-the-art in topological mapping are FABMAP [3] and CAT-SLAM [4], but limit themselves to a binary decision, i.e. whether or not the location was visited already. In the envisioned application however, it is desirable to have actual image point correspondences to facilitate bundle adjustment. An approach closer to this goal, by directly attempting to match local features among images, is described in [5]. The utilized epipolar constraint to cope with false positives is, however, not error-free. The devised approach method implements more robust error dismissal. The approach consists of two major steps. First, the issue of detecting revisited sites over time is encountered by clustering GPS information and taking its inaccuracy into account. Next, in every route of such a cluster, two poses are selected that are expected to contain common elements, based on a Naive Bayesian matching framework with severely downscaled images. When a so-called cross-route pose pair is obtained, re-occurrences of the same physical points are tracked in the associated images. This problem is treated in two steps: single pose matching and cross-route image matching. The former finds matches and deducts corresponding 3D points using the SURF [1] detector and descriptor among views taken from the same van position. Since camera calibration is available, an epipolar consistency check is straightforward. Using the surviving matches, for every van pose a point cloud results. The latter step of cross-route image matching attempts to match the images from different van poses again using SURF. This practice establishes a link between the two earlier constructed point clouds. PROSAC [2], a prioritized RANSAC algorithm, is applied to robustly calculate the transformation between the two point clouds in a time-optimal way while pruning out false positives. Figure 1 provides an illustration. This paper has two main contributions. First, our novel loop-closure technique does not depend on single image pairs for correspondences. Instead, a cloud is constructed around two van poses and these clouds are fitted together; an image-to-image approach is extended to a pose-to-pose approach. Second, our method is able to detect matches in challenging wide baseline conditions, were other systems tend to fail. Since it concerns a system-specific application, a specialized dataset is devised that comprises a substantial amount of images from an urban environment. Separate datasets were used for parameter tuning and subFigure 1: Illustration of cross-route image matches after pruning of false positives by means of PROSAC. Note that these are not the only correspondences found; also for other image combinations matches are tracked.