English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

A commonly occurring computation idiom in neural networks is to perform somepointwise operations on the result of a matrix multiplication. Such a sequenceof operations is typically represented as a computation graph in deep learningcompilers. When compiling to a GPU target, these computations can beindividually mapped to manually tuned implementations provided by librariessuch as cuBLAS and cuDNN. These libraries also provide off-the-shelf supportfor targeting tensor cores in NVIDIA GPUs, which can lead to huge performanceboosts through their specialized support for mixed-precision matrix math.Alternatively, tensor cores can be programmed directly using CUDA APIs orinline assembly instructions, which opens up the possibility of generatingefficient CUDA kernels automatically for such computations.  Automatic kernel generation is particularly crucial when it is beneficial togenerate efficient code for an entire computation graph by fusing severaloperations into a single device function instead of invoking a separate kernelfor each of them. Polyhedral compilation techniques provide a systematicapproach for the analysis and transformation of a sequence of affineloop-nests. In this paper, we describe a polyhedral approach to generateefficient CUDA kernels for matrix multiplication using inline assemblyinstructions for programming tensor cores on NVIDIA Volta GPUs. Furthermore, webuild on this approach to generate fused kernels for computation sequencesinvolving matrix multiplication and pointwise operations such as bias addition,ReLU activation etc. Experimental evaluation of these techniques show thatautomatically generated kernels can provide significantly better performancethan manually tuned library implementations, with speedups ranging up to 2.55X.

Automatic Kernel Generation for Volta Tensor Cores