English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

It has been over six years since the Transformer architecture was putforward. Surprisingly, the vanilla Transformer architecture is still widelyused today. One reason is that the lack of deep understanding and comprehensiveinterpretation of the Transformer architecture makes it more challenging toimprove the Transformer architecture. In this paper, we first interpret theTransformer architecture comprehensively in plain words based on ourunderstanding and experiences. The interpretations are further proved andverified. These interpretations also cover the Extractor, a family of drop-inreplacements for the multi-head self-attention in the Transformer architecture.Then, we propose an improvement on a type of the Extractor that outperforms theself-attention, without introducing additional trainable parameters.Experimental results demonstrate that the improved Extractor performs evenbetter, showing a way to improve the Transformer architecture.

Interpretation of the Transformer and Improvement of the Extractor