English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Running deep neural network (DNN) inference on mobile devices, i.e., mobileinference, has become a growing trend, making inference less dependent onnetwork connections and keeping private data locally. The prior studies onoptimizing DNNs for mobile inference typically focus on the metric of averageinference latency, thus implicitly assuming that mobile inference exhibitslittle latency variability. In this note, we conduct a preliminary measurementstudy on the latency variability of DNNs for mobile inference. We show that theinference latency variability can become quite significant in the presence ofCPU resource contention. More interestingly, unlike the common belief that therelative performance superiority of DNNs on one device can carry over toanother device and/or another level of resource contention, we highlight that aDNN model with a better latency performance than another model can becomeoutperformed by the other model when resource contention be more severe orrunning on another device. Thus, when optimizing DNN models for mobileinference, only measuring the average latency may not be adequate; instead,latency variability under various conditions should be accounted for, includingbut not limited to different devices and different levels of CPU resourcecontention considered in this note.

A Note on Latency Variability of Deep Neural Networks for Mobile Inference