Premium
Capturing and Understanding Workers’ Activities in Far‐Field Surveillance Videos with Deep Action Recognition and Bayesian Nonparametric Learning
Author(s) -
Luo Xiaochun,
Li Heng,
Yang Xincong,
Yu Yantao,
Cao Dongping
Publication year - 2019
Publication title -
computer‐aided civil and infrastructure engineering
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 2.773
H-Index - 82
eISSN - 1467-8667
pISSN - 1093-9687
DOI - 10.1111/mice.12419
Subject(s) - machine learning , artificial intelligence , hidden markov model , computer science , bayesian probability , field (mathematics) , nonparametric statistics , action (physics) , data mining , statistics , mathematics , physics , quantum mechanics , pure mathematics
Recording workers’ activities is an important, but burdensome, management task for site supervisors. The last decade has seen a growing trend toward vision‐based activity recognition. However, recognizing workers’ activities in far‐field surveillance videos is understudied. This study proposes a hierarchical statistical method for recognizing workers’ activities in far‐field surveillance videos. The method consists of two steps. First, a deep action recognition method was used to recognize workers’ actions, and a new fusion strategy was proposed to consider the characteristics of far‐field surveillance videos. The deep action recognition method with the new fusion strategy has achieved the comparable performance (0.84 average accuracy) on the far‐field surveillance data set in contrast to the original method on the public data sets. Second, a Bayesian nonparametric hidden semi‐Markov model was innovatively used to model and infer workers’ activities based on action sequences. The latent states of the fitted Bayesian model captured workers’ activities in terms of state duration distributions and state transition distributions, which are indispensable for understanding workers’ time allocation. It has been preliminarily illustrated that the activity information learned by the Bayesian model possesses the potential to implement objective work sampling, personal physical fatigue monitoring, trade‐level health risk assessment, and process‐based quality control. Also, the limitations of this study are discussed.