English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Due to a resource-constrained environment, network compression has become animportant part of deep neural networks research. In this paper, we propose anew compression method, \textit{Inter-Layer Weight Prediction} (ILWP) andquantization method which quantize the predicted residuals between the weightsin all convolution layers based on an inter-frame prediction method inconventional video coding schemes. Furthermore, we found a phenomenon\textit{Smoothly Varying Weight Hypothesis} (SVWH) which is that the weights inadjacent convolution layers share strong similarity in shapes and values, i.e.,the weights tend to vary smoothly along with the layers. Based on SVWH, wepropose a second ILWP and quantization method which quantize the predictedresiduals between the weights in adjacent convolution layers. Since thepredicted weight residuals tend to follow Laplace distributions with very lowvariance, the weight quantization can more effectively be applied, thusproducing more zero weights and enhancing the weight compression ratio. Inaddition, we propose a new \textit{inter-layer loss} for eliminatingnon-texture bits, which enabled us to more effectively store only texture bits.That is, the proposed loss regularizes the weights such that the collocatedweights between the adjacent two layers have the same values. Finally, wepropose an ILWP with an inter-layer loss and quantization method. Ourcomprehensive experiments show that the proposed method achieves a much higherweight compression rate at the same accuracy level compared with the previousquantization-based compression methods in deep neural networks.

An Inter-Layer Weight Prediction and Quantization for Deep Neural Networks based on a Smoothly Varying Weight Hypothesis