English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep Neural Networks(DNNs) have many parameters and activation data, andthese both are expensive to implement. One method to reduce the size of the DNNis to quantize the pre-trained model by using a low-bit expression for weightsand activations, using fine-tuning to recover the drop in accuracy. However, itis generally difficult to train neural networks which use low-bit expressions.One reason is that the weights in the middle layer of the DNN have a widedynamic range and so when quantizing the wide dynamic range into a few bits,the step size becomes large, which leads to a large quantization error andfinally a large degradation in accuracy. To solve this problem, this papermakes the following three contributions without using any additional learningparameters and hyper-parameters. First, we analyze how batch normalization,which causes the aforementioned problem, disturbs the fine-tuning of thequantized DNN. Second, based on these results, we propose a new pruning methodcalled Pruning for Quantization (PfQ) which removes the filters that disturbthe fine-tuning of the DNN while not affecting the inferred result as far aspossible. Third, we propose a workflow of fine-tuning for quantized DNNs usingthe proposed pruning method(PfQ). Experiments using well-known models anddatasets confirmed that the proposed method achieves higher performance with asimilar model size than conventional quantization methods includingfine-tuning.

Filter Pre-Pruning for Improved Fine-tuning of Quantized Deep Neural Networks