English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Designing hardware accelerators for deep neural networks (DNNs) has been muchdesired. Nonetheless, most of these existing accelerators are built for eitherconvolutional neural networks (CNNs) or recurrent neural networks (RNNs).Recently, the Transformer model is replacing the RNN in the natural languageprocessing (NLP) area. However, because of intensive matrix computations andcomplicated data flow being involved, the hardware design for the Transformermodel has never been reported. In this paper, we propose the first hardwareaccelerator for two key components, i.e., the multi-head attention (MHA)ResBlock and the position-wise feed-forward network (FFN) ResBlock, which arethe two most complex layers in the Transformer. Firstly, an efficient method isintroduced to partition the huge matrices in the Transformer, allowing the twoResBlocks to share most of the hardware resources. Secondly, the computationflow is well designed to ensure the high hardware utilization of the systolicarray, which is the biggest module in our design. Thirdly, complicatednonlinear functions are highly optimized to further reduce the hardwarecomplexity and also the latency of the entire system. Our design is coded usinghardware description language (HDL) and evaluated on a Xilinx FPGA. Comparedwith the implementation on GPU with the same setting, the proposed designdemonstrates a speed-up of 14.6x in the MHA ResBlock, and 3.4x in the FFNResBlock, respectively. Therefore, this work lays a good foundation forbuilding efficient hardware accelerators for multiple Transformer networks.

Hardware Accelerator for Multi-Head Attention and Position-Wise Feed-Forward in the Transformer