English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Recently, several studies reported that dot-product selfattention (SA) maynot be indispensable to the state-of-theart Transformer models. Motivated bythe fact that dense synthesizer attention (DSA), which dispenses with dotproducts and pairwise interactions, achieved competitive results in manylanguage processing tasks, in this paper, we first propose a DSA-based speechrecognition, as an alternative to SA. To reduce the computational complexityand improve the performance, we further propose local DSA (LDSA) to restrictthe attention scope of DSA to a local range around the current central framefor speech recognition. Finally, we combine LDSA with SA to extract the localand global information simultaneously. Experimental results on the Ai-shell1Mandarine speech recognition corpus show that the proposed LDSA-Transformerachieves a character error rate (CER) of 6.49%, which is slightly better thanthat of the SA-Transformer. Meanwhile, the LDSA-Transformer requires lesscomputation than the SATransformer. The proposed combination method not onlyachieves a CER of 6.18%, which significantly outperforms the SA-Transformer,but also has roughly the same number of parameters and computational complexityas the latter. The implementation of the multi-head LDSA is available athttps://github.com/mlxu995/multihead-LDSA.

Transformer-Based End-to-End Speech Recognition with Local Dense Synthesizer Attention