Open AccessHuman‐like evaluation method for object motion detection algorithms
Author(s) -
GuzmanPando Abimael,
ChaconMurguia Mario Ignacio,
ChaconDiaz Lucia B.
Publication year - 2020
Publication title -
iet computer vision
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.38
H-Index - 37
eISSN - 1751-9640
pISSN - 1751-9632
DOI - 10.1049/iet-cvi.2019.0997
Subject(s) - segmentation , computer science , metric (unit) , artificial intelligence , task (project management) , object detection , computer vision , image segmentation , object (grammar) , tracking (education) , video tracking , machine learning , performance metric , algorithm , pattern recognition (psychology) , engineering , psychology , pedagogy , operations management , systems engineering , management , economics
This study proposes a new method to evaluate the performance of algorithms for moving object detection (MODA) in video sequences. The proposed method is based on human performance metric intervals, instead of ideal metric values (0 or 1) which are commonly used in the literature. These intervals are proposed to establish a more reliable evaluation and comparison, and to identify areas of improvement in the evaluation of MODA. The contribution of the study includes the determination of human segmentation performance metric intervals and their comparison with state‐of‐the‐art MODA, and the evaluation of their segmentation results in a tracking task to establish the impact between performance and practical utility. Results show that human participants had difficulty with achieving a perfect segmentation score. Deep learning algorithms achieved performance above the human average, while other techniques achieved a performance between 88 and 92%. Furthermore, the authors demonstrate that algorithms not ranked at the top of the quantitative metrics worked satisfactorily in a tracking experiment; and therefore, should not be discarded for real applications.