English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Transformers' compute-intensive operations pose enormous challenges for theirdeployment in resource-constrained EdgeAI / tinyML devices. As an establishedneural network compression technique, quantization reduces the hardwarecomputational and memory resources. In particular, fixed-point quantization isdesirable to ease the computations using lightweight blocks, like adders andmultipliers, of the underlying hardware. However, deploying fully-quantizedTransformers on existing general-purpose hardware, generic AI accelerators, orspecialized architectures for Transformers with floating-point units might beinfeasible and/or inefficient.  Towards this, we propose SwiftTron, an efficient specialized hardwareaccelerator designed for Quantized Transformers. SwiftTron supports theexecution of different types of Transformers' operations (like Attention,Softmax, GELU, and Layer Normalization) and accounts for diverse scalingfactors to perform correct computations. We synthesize the complete SwiftTronarchitecture in a $65$ nm CMOS technology with the ASIC design flow. OurAccelerator executes the RoBERTa-base model in 1.83 ns, while consuming 33.64mW power, and occupying an area of 273 mm^2. To ease the reproducibility, theRTL of our SwiftTron architecture is released athttps://github.com/albertomarchisio/SwiftTron.

SwiftTron: An Efficient Hardware Accelerator for Quantized Transformers