English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Transformer has been widely-used in many Natural Language Processing (NLP)tasks and the scaled dot-product attention between tokens is a core module ofTransformer. This attention is a token-wise design and its complexity isquadratic to the length of sequence, limiting its application potential forlong sequence tasks. In this paper, we propose a dimension-wise attentionmechanism based on which a novel language modeling approach (namelyTensorCoder) can be developed. The dimension-wise attention can reduce theattention complexity from the original $O(N^2d)$ to $O(Nd^2)$, where $N$ is thelength of the sequence and $d$ is the dimensionality of head. We verifyTensorCoder on two tasks including masked language modeling and neural machinetranslation. Compared with the original Transformer, TensorCoder not onlygreatly reduces the calculation of the original model but also obtains improvedperformance on masked language modeling task (in PTB dataset) and comparableperformance on machine translation tasks.

TensorCoder: Dimension-Wise Attention via Tensor Representation for Natural Language Modeling