Shape-based Image Correspondence

Current state-of-the-art dense correspondence algorithms establish correspondences between pair of images by searching for a flow field that minimizes the distance between local signatures (e.g., color histogram, SIFT descriptor) of aligned pixels while preserving smoothness. Agnostic to the global signatures (e.g., object membership, category of object), these local signatures face difficulties in resolving alignment ambiguities when scene content undergoes type and configuration variation. In this paper, we investigate the effect of adding shape correspondence constraints either in the form of pair of corresponding contour fragments or pair of closed curves. We find the shape does not play a significant role in optical flow and stereo correspondence but it does play a significant role when scene content changes are large. We also explore using object proposals as a way of providing shape constraints with encouraging results.