Unsupervised extraction of coherent regions for image based rendering

Image based rendering using undersampled light fields suffe rs from aliasing effects. These effects can be drastically reduced by usi ng some geometric information. In pop-up light field rendering [18], the scene is segmented into coherent layers, usually corresponding to approximately planar regions, that can be rendered free of aliasing. As opposed to the supervised method in the pop-up light field, we propose an unsupervised extractio n of coherent regions. The problem is posed in a multidimensional variational framework using the level set method [16]. Since the segmentation is done jointly over all the images, coherence can be imposed throughout the data. However, instead of using active hypersurfaces, we derive a semi-parametric methodology that takes into account the constraints imposed by the camera setup and the occlusion ordering. The resulting framework is a global multidimensional region competition that is consistent in all the imag es and efficiently handles occlusions. We show the validity of the method with some captured multi-view datasets. Other special effects by coherent reg ion manipulation are also demonstrated.