English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

The design choices in Transformer feed-forward neural networks have resultedin significant computational and parameter overhead. In this work, we emphasizethe importance of hidden dimensions in designing lightweight FFNs, a factoroften overlooked in previous architectures. Guided by this principle, weintroduce PartialFormer, a parameter-efficient Transformer architectureutilizing multiple smaller FFNs to reduce parameters and computation whilemaintaining essential hidden dimensions. These smaller FFNs are integrated intoa multi-head attention mechanism for effective collaboration. We also propose atailored head scaling strategy to enhance PartialFormer's capabilities.Furthermore, we present a residual-like attention calculation to improve depthscaling within PartialFormer. Extensive experiments on 9 translation tasks and1 abstractive summarization task validate the effectiveness of ourPartialFormer approach on machine translation and summarization tasks. Our codewould be available at: https://github.com/zhengkid/PartialFormer.

PartialFormer: Modeling Part Instead of Whole for Machine Translation