English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Recently, self-supervised pre-training has shown significant improvements inmany areas of machine learning, including speech and NLP. We propose usinglarge self-supervised pre-trained models for both audio and text modality withcross-modality attention for multimodal emotion recognition. We use Wav2Vec2.0[1] as an audio encoder base for robust speech features extraction and the BERTmodel [2] as a text encoder base for better contextual representation of text.These high capacity models trained on large amounts of unlabeled data containrich feature representations and improve the downstream task's performance. Weuse the cross-modal attention [3] mechanism to learn alignment between audioand text representations from self-supervised models. Cross-modal attentionalso helps in extracting interactive information between audio and textfeatures. We obtain utterance-level feature representation from frame-levelfeatures using statistics pooling for both audio and text modality and combinethem using the early fusion technique. Our experiments show that the proposedapproach obtains a 1.88% absolute improvement in accuracy compared to theprevious state-of-the-art method [3] on the IEMOCAP dataset [35]. We alsoconduct unimodal experiments for both audio and text modalities and comparethem with previous best methods.

Using Large Pre-Trained Models with Cross-Modal Attention for Multi-Modal Emotion Recognition