Open AccessHierarchical multi-view aggregation network for sensor-based human activity recognition
Author(s) -
Xiheng Zhang,
Yongkang Wong,
Mohan Kankanhalli,
Geng Wang
Publication year - 2019
Publication title -
plos one
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.99
H-Index - 332
ISSN - 1932-6203
DOI - 10.1371/journal.pone.0221390
Subject(s) - computer science , artificial intelligence , leverage (statistics) , activity recognition , feature (linguistics) , benchmark (surveying) , pattern recognition (psychology) , context (archaeology) , construct (python library) , representation (politics) , modality (human–computer interaction) , machine learning , paleontology , philosophy , linguistics , geodesy , politics , political science , law , biology , programming language , geography
Sensor-based human activity recognition aims at detecting various physical activities performed by people with ubiquitous sensors. Different from existing deep learning-based method which mainly extracting black-box features from the raw sensor data, we propose a hierarchical multi-view aggregation network based on multi-view feature spaces. Specifically, we first construct various views of feature spaces for each individual sensor in terms of white-box features and black-box features. Then our model learns a unified representation for multi-view features by aggregating views in a hierarchical context from the aspect of feature level, position level and modality level. We design three aggregation modules corresponding to each level aggregation respectively. Based on the idea of non-local operation and attention, our fusion method is able to capture the correlation between features and leverage the relationship across different sensor position and modality. We comprehensively evaluate our method on 12 human activity benchmark datasets and the resulting accuracy outperforms the state-of-the-art approaches.