English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

How can we effectively inform content selection in Transformer-basedabstractive summarization models? In this work, we present asimple-yet-effective attention head masking technique, which is applied onencoder-decoder attentions to pinpoint salient content at inference time. Usingattention head masking, we are able to reveal the relation betweenencoder-decoder attentions and content selection behaviors of summarizationmodels. We then demonstrate its effectiveness on three document summarizationdatasets based on both in-domain and cross-domain settings. Importantly, ourmodels outperform prior state-of-the-art models on CNN/Daily Mail and New YorkTimes datasets. Moreover, our inference-time masking technique is alsodata-efficient, requiring only 20% of the training samples to outperform BARTfine-tuned on the full CNN/DailyMail dataset.

Attention Head Masking for Inference Time Content Selection in Abstractive Summarization