Automatic layered RGB‐D scene flow estimation with optical flow field constraint

Scene flow estimation with RGB‐D frames is receiving increasing attention in digital video processing and computer vision due to the widespread use of depth sensors. Existing methods based on object segmentation have shown their effectiveness for object occlusion and large displacement. However, improper segmentation often causes incomplete segmented areas or incorrect edges, which will result in inaccurate occlusion inference and scene flow estimation. To this end, an automatic layered RGB‐D scene flow estimation method is proposed, which achieves more accurate layering of objects in depth image by exploring motion information. The authors exploit super‐pixel segmentation for initial layering, which is beneficial for preserving edges and integrity of the objects. Furthermore, an optical flow which is highly correlated with the scene is also used for automatic layering. Finally, the utilisation of super‐pixel segmentation and motion information would ensure the integrity of the object area and improve the accuracy of scene flow. They have validated their approach both qualitatively and quantitatively on several public datasets. Experimental results show that the proposed method is able to preserve the integrity of the object and achieve lower root mean square error and average angular error as compared with the current state‐of‐the‐art algorithms.