Multiple motion and occlusion segmentation with a multiphase level set method

In this paper, we propose a new variational formulation for simultaneous multiple motion segmentation and occlusion detection in an image sequence. For the representation of segmented regions, we use the multiphase level set method proposed by Vese and Chan. This method allows an ecien t representation of up to 2L regions with L level-set functions. Moreover, by construction, it enforces a domain partition with no gaps and overlaps. This is unlike previous variational approaches to multiple motion segmentation, where additional constraints were needed. The variational framework we propose can incorporate an arbitrary number of motion transformations as well as occlusion areas. In order to minimize the resulting energy, we developed a two-step algorithm. In the rst step, we use a feature-based method to estimate the motions present in the image sequence. In the second step, based on the extracted motion information, we iteratively evolve all level set functions in the gradient descent direction to nd the nal segmentation. We have tested the above algorithm on both synthetic- and natural-motion data with very promising results. We show here segmentation results for two real video sequences.