English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep learning (DL) frameworks take advantage of GPUs to improve the speed ofDL inference and training. Ideally, DL frameworks should be able to fullyutilize the computation power of GPUs such that the running time depends on theamount of computation assigned to GPUs. Yet, we observe that in scheduling GPUtasks, existing DL frameworks suffer from inefficiencies such as largescheduling overhead and unnecessary serial execution. To this end, we proposeNimble, a DL execution engine that runs GPU tasks in parallel with minimalscheduling overhead. Nimble introduces a novel technique called ahead-of-time(AoT) scheduling. Here, the scheduling procedure finishes before executing theGPU kernel, thereby removing most of the scheduling overhead during run time.Furthermore, Nimble automatically parallelizes the execution of GPU tasks byexploiting multiple GPU streams in a single GPU. Evaluation on a variety ofneural networks shows that compared to PyTorch, Nimble speeds up inference andtraining by up to 22.34$\times$ and 3.61$\times$, respectively. Moreover,Nimble outperforms state-of-the-art inference systems, TensorRT and TVM, by upto 2.81$\times$ and 1.70$\times$, respectively.

Nimble: Lightweight and Parallel GPU Task Scheduling for Deep Learning