English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

It is appealing but challenging to achieve real-time deep neural network(DNN) inference on mobile devices because even the powerful modern mobiledevices are considered as ``resource-constrained'' when executing large-scaleDNNs. It necessitates the sparse model inference via weight pruning, i.e., DNNweight sparsity, and it is desirable to design a new DNN weight sparsity schemethat can facilitate real-time inference on mobile devices while preserving ahigh sparse model accuracy. This paper designs a novel mobile inferenceacceleration framework GRIM that is General to both convolutional neuralnetworks (CNNs) and recurrent neural networks (RNNs) and that achievesReal-time execution and high accuracy, leveraging fine-grained structuredsparse model Inference and compiler optimizations for Mobiles. We start byproposing a new fine-grained structured sparsity scheme through the Block-basedColumn-Row (BCR) pruning. Based on this new fine-grained structured sparsity,our GRIM framework consists of two parts: (a) the compiler optimization andcode generation for real-time mobile inference; and (b) the BCR pruningoptimizations for determining pruning hyperparameters and performing weightpruning. We compare GRIM with Alibaba MNN, TVM, TensorFlow-Lite, a sparseimplementation based on CSR, PatDNN, and ESE (a representative FPGA inferenceacceleration framework for RNNs), and achieve up to 14.08x speedup.

GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices based on Fine-Grained Structured Weight Sparsity