English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Multimodal fusion frameworks for Human Action Recognition (HAR) using depthand inertial sensor data have been proposed over the years. In most of theexisting works, fusion is performed at a single level (feature level ordecision level), missing the opportunity to fuse rich mid-level featuresnecessary for better classification. To address this shortcoming, in thispaper, we propose three novel deep multilevel multimodal fusion frameworks tocapitalize on different fusion strategies at various stages and to leverage thesuperiority of multilevel fusion. At input, we transform the depth data intodepth images called sequential front view images (SFIs) and inertial sensordata into signal images. Each input modality, depth and inertial, is furthermade multimodal by taking convolution with the Prewitt filter. Creating"modality within modality" enables further complementary and discriminativefeature extraction through Convolutional Neural Networks (CNNs). CNNs aretrained on input images of each modality to learn low-level, high-level andcomplex features. Learned features are extracted and fused at different stagesof the proposed frameworks to combine discriminative and complementaryinformation. These highly informative features are served as input to amulti-class Support Vector Machine (SVM). We evaluate the proposed frameworkson three publicly available multimodal HAR datasets, namely, UTD MultimodalHuman Action Dataset (MHAD), Berkeley MHAD, and UTD-MHAD Kinect V2.Experimental results show the supremacy of the proposed fusion frameworks overexisting methods.

Human Action Recognition Using Deep Multilevel Multimodal (M2) Fusion of Depth and Inertial Sensors