English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

NVIDIA Tensor Core is a mixed-precision matrix-matrix multiplication andaddition computing unit, where the theoretical peak performance is more than300 TFlop/s on NVIDIA A100 GPU. NVIDIA provides WMMA API for using Tensor Coresin custom kernel functions. The most common way to use Tensor Core is to supplythe input matrices from shared memory, which has higher bandwidth than globalmemory. However, the Bytes-per-Flops (B/F) ratio of the shared memory andTensor Cores is small since the performance of Tensor Cores is high. Thus, itis important to reduce the shared memory footprint for efficient Tensor Coresusage. In this paper, we analyze the simple matrix-matrix multiplication onTensor Cores by the roofline model and figure out that the bandwidth of sharedmemory might be a limitation of the performance when using WMMA API. Toalleviate this issue, we provide a WMMA API extension library to boost thethroughput of the computation, which has two components. The first one allowsfor manipulating the array of registers input to Tensor Cores flexibly. Weevaluate the performance improvement of this library. The outcome of ourevaluation shows that our library reduces the shared memory footprint andspeeds up the computation using Tensor Cores. The second one is an API for theSGEMM emulation on Tensor Cores without additional shared memory usage. We havedemonstrated that the single-precision emulating batch SGEMM implementation onTensor Cores using this library achieves 54.2 TFlop/s on A100 GPU, whichoutperforms the theoretical peak performance of FP32 SIMT Cores while achievingthe same level of accuracy as cuBLAS. The achieved throughput can not beachieved without reducing the shared memory footprint done by our library withthe same amount of register usage.

Reducing shared memory footprint to leverage high throughput on Tensor Cores and its flexible API extension library