English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Multimodal Emotion Recognition refers to the classification of input videosequences into emotion labels based on multiple input modalities (usuallyvideo, audio and text). In recent years, Deep Neural networks have shownremarkable performance in recognizing human emotions, and are on par withhuman-level performance on this task. Despite the recent advancements in thisfield, emotion recognition systems are yet to be accepted for real world setupsdue to the obscure nature of their reasoning and decision-making process. Mostof the research in this field deals with novel architectures to improve theperformance for this task, with a few attempts at providing explanations forthese models' decisions. In this paper, we address the issue ofinterpretability for neural networks in the context of emotion recognitionusing Concept Activation Vectors (CAVs). To analyse the model's latent space,we define human-understandable concepts specific to Emotion AI and map them tothe widely-used IEMOCAP multimodal database. We then evaluate the influence ofour proposed concepts at multiple layers of the Bi-directional Contextual LSTM(BC-LSTM) network to show that the reasoning process of neural networks foremotion recognition can be represented using human-understandable concepts.Finally, we perform hypothesis testing on our proposed concepts to show thatthey are significant for interpretability of this task.

Interpretability for Multimodal Emotion Recognition using Concept Activation Vectors