English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Modern deep learning applications urge to push the model inference takingplace at the edge devices for multiple reasons such as achieving shorterlatency, relieving the burden of the network connecting to the cloud, andprotecting user privacy. The Convolutional Neural Network (\emph{CNN}) is oneof the most widely used model family in the applications. Given the highcomputational complexity of the CNN models, it is favorable to execute them onthe integrated GPUs at the edge devices, which are ubiquitous and have morepower and better energy efficiency than the accompanying CPUs. However,programming on integrated GPUs efficiently is challenging due to the variety oftheir architectures and programming interfaces. This paper proposes anend-to-end solution to execute CNN model inference on the integrated GPUs atthe edge, which uses a unified IR to represent and optimize vision-specificoperators on integrated GPUs from multiple vendors, as well as leveragesmachine learning-based scheduling search schemes to optimizecomputationally-intensive operators like convolution. Our solution evenprovides a fallback mechanism for operators not suitable or convenient to runon GPUs. The evaluation results suggest that compared to state-of-the-artsolutions backed up by the vendor-provided high-performance libraries on IntelGraphics, ARM Mali GPU, and Nvidia integrated Maxwell GPU, our solutionachieves similar, or even better (up to 1.62$\times$), performance on a numberof popular image classification and object detection models. In addition, oursolution has a wider model coverage and is more flexible to embrace new models.Our solution has been adopted in production services in AWS and isopen-sourced.

A Unified Optimization Approach for CNN Model Inference on Integrated GPUs