English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The Transformer-based models with the multi-head self-attention mechanism arewidely used in natural language processing, and provide state-of-the-artresults. While the pre-trained language backbones are shown to implicitlycapture certain linguistic knowledge, explicitly incorporating structure-awarefeatures can bring about further improvement on the downstream tasks. However,such enhancement often requires additional neural components and increasestraining parameter size. In this work, we investigate the attention headselection and manipulation strategy for feature injection from a networkpruning perspective, and conduct a case study on dialogue summarization. Wefirst rank attention heads in a Transformer-based summarizer with layer-wiseimportance. We then select the underused heads through extensive analysis, andinject structure-aware features by manipulating the selected heads.Experimental results show that the importance-based head selection is effectivefor feature injection, and dialogue summarization can be improved byincorporating coreference information via head manipulation.

Picking the Underused Heads: A Network Pruning Perspective of Attention Head Selection for Fusing Dialogue Coreference Information