English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Learning algorithms become more powerful, often at the cost of increasedcomplexity. In response, the demand for algorithms to be transparent isgrowing. In NLP tasks, attention distributions learned by attention-based deeplearning models are used to gain insights in the models' behavior. To whichextent is this perspective valid for all NLP tasks? We investigate whetherdistributions calculated by different attention heads in a transformerarchitecture can be used to improve transparency in the task of abstractivesummarization. To this end, we present both a qualitative and quantitativeanalysis to investigate the behavior of the attention heads. We show that someattention heads indeed specialize towards syntactically and semanticallydistinct input. We propose an approach to evaluate to which extent theTransformer model relies on specifically learned attention distributions. Wealso discuss what this implies for using attention distributions as a means oftransparency.

Do Transformer Attention Heads Provide Transparency in Abstractive Summarization?