English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

There has been significant debate in the NLP community about whether or notattention weights can be used as an explanation - a mechanism for interpretinghow important each input token is for a particular prediction. The validity of"attention as explanation" has so far been evaluated by computing the rankcorrelation between attention-based explanations and existing featureattribution explanations using LSTM-based models. In our work, we (i) comparethe rank correlation between five more recent feature attribution methods andtwo attention-based methods, on two types of NLP tasks, and (ii) extend thisanalysis to also include transformer-based models. We find that attention-basedexplanations do not correlate strongly with any recent feature attributionmethods, regardless of the model or task. Furthermore, we find that none of thetested explanations correlate strongly with one another for thetransformer-based model, leading us to question the underlying assumption thatwe should measure the validity of attention-based explanations based on howwell they correlate with existing feature attribution explanation methods.After conducting experiments on five datasets using two different models, weargue that the community should stop using rank correlation as an evaluationmetric for attention-based explanations. We suggest that researchers andpractitioners should instead test various explanation methods and employ ahuman-in-the-loop process to determine if the explanations align with humanintuition for the particular use case at hand.

A Song of (Dis)agreement: Evaluating the Evaluation of Explainable Artificial Intelligence in Natural Language Processing