Open AccessAttention‐based video object segmentation algorithm
Author(s) -
Cao Ying,
Sun Lijuan,
Han Chong,
Guo Jian
Publication year - 2021
Publication title -
iet image processing
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.401
H-Index - 45
eISSN - 1751-9667
pISSN - 1751-9659
DOI - 10.1049/ipr2.12135
Subject(s) - computer science , artificial intelligence , optical flow , computer vision , segmentation , residual , frame (networking) , motion estimation , object (grammar) , coherence (philosophical gambling strategy) , scheme (mathematics) , image segmentation , motion (physics) , pattern recognition (psychology) , image (mathematics) , algorithm , mathematics , telecommunications , mathematical analysis , statistics
To improve the segmentation performance on videos with large object motion or deformation, a novel scheme is proposed which has two branches. In one branch, the attention mechanism is first utilized to highlight objects‐related features. Then, to well consider the temporal coherence of videos, Conv3D is integrated to capture short‐term temporal features, and the designed attention residual convolutional long–short‐term memory is adopted to capture the long–short‐term temporal information of objects under the interference of redundant video frames. Meanwhile, considering the negative effect of background motion, in another branch, the optical flow‐based prediction model is introduced to predict objects regions in subsequent video frames with the annotated initial frame. At last, based on the fused results of two branches, the global thresholds and noising area clean method are employed to obtain segmented objects. The experiments on DAVIS2016 and CDnet2014 exhibit the competitive performance of the proposed scheme.