English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Transformers are considered one of the most important deep learning modelssince 2018, in part because it establishes state-of-the-art (SOTA) records andcould potentially replace existing Deep Neural Networks (DNNs). Despite theremarkable triumphs, the prolonged turnaround time of Transformer models is awidely recognized roadblock. The variety of sequence lengths imposes additionalcomputing overhead where inputs need to be zero-padded to the maximum sentencelength in the batch to accommodate the parallel computing platforms. This papertargets the field-programmable gate array (FPGA) and proposes a coherentsequence length adaptive algorithm-hardware co-design for Transformeracceleration. Particularly, we develop a hardware-friendly sparse attentionoperator and a length-aware hardware resource scheduling algorithm. Theproposed sparse attention operator brings the complexity of attention-basedmodels down to linear complexity and alleviates the off-chip memory traffic.The proposed length-aware resource hardware scheduling algorithm dynamicallyallocates the hardware resources to fill up the pipeline slots and eliminatesbubbles for NLP tasks. Experiments show that our design has very small accuracyloss and has 80.2 $\times$ and 2.6 $\times$ speedup compared to CPU and GPUimplementation, and 4 $\times$ higher energy efficiency than state-of-the-artGPU accelerator optimized via CUBLAS GEMM.

A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining