English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Neural attention, especially the self-attention made popular by theTransformer, has become the workhorse of state-of-the-art natural languageprocessing (NLP) models. Very recent work suggests that the self-attention inthe Transformer encodes syntactic information; Here, we show thatself-attention scores encode semantics by considering sentiment analysis tasks.In contrast to gradient-based feature attribution methods, we propose a simpleand effective Layer-wise Attention Tracing (LAT) method to analyze structuredattention weights. We apply our method to Transformer models trained on twotasks that have surface dissimilarities, but share common semantics---sentimentanalysis of movie reviews and time-series valence prediction in life storynarratives. Across both tasks, words with high aggregated attention weightswere rich in emotional semantics, as quantitatively validated by an emotionlexicon labeled by human annotators. Our results show that structured attentionweights encode rich semantics in sentiment analysis, and match humaninterpretations of semantics.

Structured Self-Attention Weights Encode Semantics in Sentiment Analysis