Multi-view Human Action Recognition Based on TSN Architecture Integrated with GRU

Human action recognition is one of the most important problems in computer vision. In this paper, we propose a novel architecture for multi-view human action recognition. The proposal exploits the temporal features and fuses the information from different camera views. Based on the idea of TSN (Temporal Segment Networks) which is working with segments of videos, we recommend aggregating scores from segments by an RNN (Recurrent Neural Network) module to enhance the quality of dynamics features. Furthermore, the proposed architecture is designed to form a multi-branch network with each branch taking responsibility for extracting a view-specific information, and the final feature is formed by combining results from branches lastly. Experiments on two datasets NUMA and MicaHandGesture have proved that the proposed architecture works effectively in different scenarios. Our model has achieved promising results with the accuracy slightly surpass other previous works.