English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Deep Neural Networks (DNNs) have shown excellent performance in a wide rangeof machine learning applications. Knowing the latency of running a DNN model ortensor program on a specific device is useful in various tasks, such as DNNgraph- or tensor-level optimization and device selection. Considering the largespace of DNN models and devices that impede direct profiling of allcombinations, recent efforts focus on building a predictor to model theperformance of DNN models on different devices. However, none of the existingattempts have achieved a cost model that can accurately predict the performanceof various tensor programs while supporting both training and inferenceaccelerators. We propose CDMPP, an efficient tensor program latency predictionframework for both cross-model and cross-device prediction. We design aninformative but efficient representation of tensor programs, called compactASTs, and a pre-order-based positional encoding method, to capture the internalstructure of tensor programs. We develop a domain-adaption-inspired method tolearn domain-invariant representations and devise a KMeans-based samplingalgorithm, for the predictor to learn from different domains (i.e., differentDNN operators and devices). Our extensive experiments on a diverse range of DNNmodels and devices demonstrate that CDMPP significantly outperformsstate-of-the-art baselines with 14.03% and 10.85% prediction error forcross-model and cross-device prediction, respectively, and one order ofmagnitude higher training efficiency. The implementation and the expandeddataset are available at https://github.com/joapolarbear/cdmpp.

CDMPP: A Device-Model Agnostic Framework for Latency Prediction of Tensor Programs