English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

The recent advances in deep neural networks (DNNs) make them attractive forembedded systems. However, it can take a long time for DNNs to make aninference on resource-constrained computing devices. Model compressiontechniques can address the computation issue of deep inference on embeddeddevices. This technique is highly attractive, as it does not rely onspecialized hardware, or computation-offloading that is often infeasible due toprivacy concerns or high latency. However, it remains unclear how modelcompression techniques perform across a wide range of DNNs. To design efficientembedded deep learning solutions, we need to understand their behaviors. Thiswork develops a quantitative approach to characterize model compressiontechniques on a representative embedded deep learning architecture, the NVIDIAJetson Tx2. We perform extensive experiments by considering 11 influentialneural network architectures from the image classification and the naturallanguage processing domains. We experimentally show that how two mainstreamcompression techniques, data quantization and pruning, perform on these networkarchitectures and the implications of compression techniques to the modelstorage size, inference time, energy consumption and performance metrics. Wedemonstrate that there are opportunities to achieve fast deep inference onembedded systems, but one must carefully choose the compression settings. Ourresults provide insights on when and how to apply model compression techniquesand guidelines for designing efficient embedded deep learning systems.

To Compress, or Not to Compress: Characterizing Deep Learning Model Compression for Embedded Inference