English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Despite the superb performance of State-Of-The-Art (SOTA) DNNs, theincreasing computational cost makes them very challenging to meet real-timelatency and accuracy requirements. Although DNN runtime latency is dictated bymodel property (e.g., architecture, operations), hardware property (e.g.,utilization, throughput), and more importantly, the effective mapping betweenthese two, many existing approaches focus only on optimizing model propertysuch as FLOPS reduction and overlook the mismatch between DNN model andhardware properties. In this work, we show that the mismatch between the variedDNN computation workloads and GPU capacity can cause the idle GPU tail effect,leading to GPU under-utilization and low throughput. As a result, the FLOPsreduction cannot bring effective latency reduction, which causes sub-optimalaccuracy versus latency trade-offs. Motivated by this, we propose a GPUruntime-aware DNN optimization methodology to eliminate such GPU tail effectadaptively on GPU platforms. Our methodology can be applied on top of existingSOTA DNN optimization approaches to achieve better latency and accuracytrade-offs. Experiments show 11%-27% latency reduction and 2.5%-4.0% accuracyimprovement over several SOTA DNN pruning and NAS methods, respectively

Towards Latency-aware DNN Optimization with GPU Runtime Analysis and Tail Effect Elimination