Wide baseline feature matching using the cross-epipolar ordering constraint

Robust feature matching across different views of the same scene taken by two cameras with wide baseline and arbitrary rotation is still an open problem. Matching based on appearance alone is unreliable, because a surface point changes its appearance depending on the viewpoint. As a result, this approach may generate "unrealizable" correspondence sets, that is, sets of pairwise correspondences that are not consistent with the epipolar geometry. We propose a novel technique, which is applicable when the epipolar lines in the two images are approximately parallel. Our algorithm only searches in the space of realizable matchings, thereby reducing the likelihood of mismatches as well as the dimension of the search domain. This approach can use any given feature descriptor with an associated distance function, and assumes no knowledge of the intrinsic parameters of the two cameras. The extension to the general case of epipoles infinite position is the object of current research.