English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The multi-head self-attention of popular transformer models is widely usedwithin Natural Language Processing (NLP), including for the task of extractivesummarization. With the goal of analyzing and pruning the parameter-heavyself-attention mechanism, there are multiple approaches proposing moreparameter-light self-attention alternatives. In this paper, we present a novelparameter-lean self-attention mechanism using discourse priors. Our new treeself-attention is based on document-level discourse information, extending therecently proposed "Synthesizer" framework with another lightweight alternative.We show empirical results that our tree self-attention approach achievescompetitive ROUGE-scores on the task of extractive summarization. When comparedto the original single-head transformer model, the tree attention approachreaches similar performance on both, EDU and sentence level, despite thesignificant reduction of parameters in the attention component. We furthersignificantly outperform the 8-head transformer model on sentence level whenapplying a more balanced hyper-parameter setting, requiring an order ofmagnitude less parameters.

Do We Really Need That Many Parameters In Transformer For Extractive Summarization? Discourse Can Help !