Learning geometrical transforms between multi camera views using Canonical Correlation Analysis

We present an unsupervised and sampling-free approach to learn the correspondence relations between pairs of cameras in closed form employing a linear model known as Canonical Correlation Analysis (CCA). The only assumption we make is that the relative orientation between the cameras involved is fixed. In a two stage algorithm, we first learn the inter-image transformation based on CCA. This analysis usually has to be done in a multi-scale framework, as applying CCA directly to full resolution images may be computationally prohibitive. In the second stage we employ the learnt transformation which is given only implicitly and predict for a given pixel in a first view its corresponding region within a second view. We denote these regions as correspondence prior. CCA has been introduced by Hotelling [2] as a method of analyzing the relations between two sets of variates and can be applied in closed form. Consider two random vectors x and y where x 2 R N and y 2 R M . The goal of CCA is to find basis vectors for which the correlation between x and y when projected onto the basis vectors are mutually maximized [3]. In the case of a single pair of basis vectors u 2 R N ,v 2 R M the projections are given as a=u T x and b=v T y. Assuming E[x]=E[y]=0, the correlation r between a and b can be written as: