Open AccessWeakly supervised video action localisation via two‐stream action activation network
Author(s) -
Yin Chang,
Liao Zhongke,
Hu Haifeng,
Chen Dihu
Publication year - 2019
Publication title -
electronics letters
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.375
H-Index - 146
ISSN - 1350-911X
DOI - 10.1049/el.2019.2088
Subject(s) - computer science , action (physics) , class (philosophy) , artificial intelligence , variance (accounting) , simple (philosophy) , pattern recognition (psychology) , state (computer science) , machine learning , algorithm , physics , quantum mechanics , philosophy , accounting , epistemology , business
This Letter introduces a weakly supervised method for human action localisation, dubbed Two‐stream Action Activation Network (TAAN). In order to generate both action category predictions and temporal location predictions only by video‐level annotations, TAAN firstly utilises class response variance to select representative segments as video‐level expressions. Then under the supervision of video‐level annotations, these expressions are used to train a classification network. Next, temporal class activation maps (T‐CAMs) are generated based on the classification response, and finally temporal level predictions are given according to T‐CAMs. In addition, the proposed method fuses two‐steam predictions in a simple but effective way to further improve the precision of the predictions. Evaluations on THUMOS 2014 dataset show TAAN outperforms state‐of‐the‐art weakly supervised methods.