English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

To fluently collaborate with people, robots need the ability to recognizehuman activities accurately. Although modern robots are equipped with varioussensors, robust human activity recognition (HAR) still remains a challengingtask for robots due to difficulties related to multimodal data fusion. Toaddress these challenges, in this work, we introduce a deep neuralnetwork-based multimodal HAR algorithm, HAMLET. HAMLET incorporates ahierarchical architecture, where the lower layer encodes spatio-temporalfeatures from unimodal data by adopting a multi-head self-attention mechanism.We develop a novel multimodal attention mechanism for disentangling and fusingthe salient unimodal features to compute the multimodal features in the upperlayer. Finally, multimodal features are used in a fully connect neural-networkto recognize human activities. We evaluated our algorithm by comparing itsperformance to several state-of-the-art activity recognition algorithms onthree human activity datasets. The results suggest that HAMLET outperformed allother evaluated baselines across all datasets and metrics tested, with thehighest top-1 accuracy of 95.12% and 97.45% on the UTD-MHAD [1] and theUT-Kinect [2] datasets respectively, and F1-score of 81.52% on the UCSD-MIT [3]dataset. We further visualize the unimodal and multimodal attention maps, whichprovide us with a tool to interpret the impact of attention mechanismsconcerning HAR.

HAMLET: A Hierarchical Multimodal Attention-based Human Activity Recognition Algorithm