Multiview video depth estimation with spatial-temporal consistency

In this paper, we present an approach to recover both spatially and temporally consistent depth maps from multiview synchronized and calibrated video streams. Depth maps are initialized by combining left-right view matching and color based segmentation. Then the color constancy and spatial coherence are integrated into the optimization framework in order to guarantee the spatial consistency at single time instant. Finally, we incorporate the depth and motion information in the form of spatial-temporal consistency constraint to refine and stabilize the depth video, without ruining the original spatial consistency in the estimation of each single instant. The experiments on different multiview sequences demonstrate the effectiveness of our method in providing both stable and accurate multiview depth estimation.