English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This work proposes a novel Deep Neural Network (DNN) quantization framework,namely RMSMP, with a Row-wise Mixed-Scheme and Multi-Precision approach.Specifically, this is the first effort to assign mixed quantization schemes andmultiple precisions within layers -- among rows of the DNN weight matrix, forsimplified operations in hardware inference, while preserving accuracy.Furthermore, this paper makes a different observation from the prior work thatthe quantization error does not necessarily exhibit the layer-wise sensitivity,and actually can be mitigated as long as a certain portion of the weights inevery layer are in higher precisions. This observation enables layer-wiseuniformality in the hardware implementation towards guaranteed inferenceacceleration, while still enjoying row-wise flexibility of mixed schemes andmultiple precisions to boost accuracy. The candidates of schemes and precisionsare derived practically and effectively with a highly hardware-informativestrategy to reduce the problem search space. With the offline determined ratioof different quantization schemes and precisions for all the layers, the RMSMPquantization algorithm uses the Hessian and variance-based method toeffectively assign schemes and precisions for each row. The proposed RMSMP istested for the image classification and natural language processing (BERT)applications and achieves the best accuracy performance among state-of-the-artsunder the same equivalent precisions. The RMSMP is implemented on FPGA devices,achieving 3.65x speedup in the end-to-end inference time for ResNet-18 onImageNet, compared with the 4-bit Fixed-point baseline.

RMSMP: A Novel Deep Neural Network Quantization Framework with Row-wise Mixed Schemes and Multiple Precisions