English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Attention mechanisms have played a crucial role in the development of complexarchitectures such as Transformers in natural language processing. However,Transformers remain hard to interpret and are considered as black-boxes. Thispaper aims to assess how attention coefficients from Transformers can help inproviding interpretability. A new attention-based interpretability methodcalled CLaSsification-Attention (CLS-A) is proposed. CLS-A computes aninterpretability score for each word based on the attention coefficientdistribution related to the part specific to the classification task within theTransformer architecture. A human-grounded experiment is conducted to evaluateand compare CLS-A to other interpretability methods. The experimental protocolrelies on the capacity of an interpretability method to provide explanation inline with human reasoning. Experiment design includes measuring reaction timesand correct response rates by human subjects. CLS-A performs comparably tousual interpretability methods regarding average participant reaction time andaccuracy. The lower computational cost of CLS-A compared to otherinterpretability methods and its availability by design within the classifiermake it particularly interesting. Data analysis also highlights the linkbetween the probability score of a classifier prediction and adequateexplanations. Finally, our work confirms the relevancy of the use of CLS-A andshows to which extent self-attention contains rich information to explainTransformer classifiers.

Evaluating self-attention interpretability through human-grounded experimental protocol