English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Impressive performance of Transformer has been attributed to self-attention,where dependencies between entire input in a sequence are considered at everyposition. In this work, we reform the neural $n$-gram model, which focuses ononly several surrounding representations of each position, with the multi-headmechanism as in Vaswani et al.(2017). Through experiments onsequence-to-sequence tasks, we show that replacing self-attention inTransformer with multi-head neural $n$-gram can achieve comparable or betterperformance than Transformer. From various analyses on our proposed method, wefind that multi-head neural $n$-gram is complementary to self-attention, andtheir combinations can further improve performance of vanilla Transformer.

Are Neighbors Enough? Multi-Head Neural n-gram can be Alternative to Self-attention