English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Weight and activation binarization is an effective approach to deep neuralnetwork compression and can accelerate the inference by leveraging bitwiseoperations. Although many binarization methods have improved the accuracy ofthe model by minimizing the quantization error in forward propagation, thereremains a noticeable performance gap between the binarized model and thefull-precision one. Our empirical study indicates that the quantization bringsinformation loss in both forward and backward propagation, which is thebottleneck of training accurate binary neural networks. To address theseissues, we propose an Information Retention Network (IR-Net) to retain theinformation that consists in the forward activations and backward gradients.IR-Net mainly relies on two technical contributions: (1) Libra ParameterBinarization (Libra-PB): simultaneously minimizing both quantization error andinformation loss of parameters by balanced and standardized weights in forwardpropagation; (2) Error Decay Estimator (EDE): minimizing the information lossof gradients by gradually approximating the sign function in backwardpropagation, jointly considering the updating ability and accurate gradients.We are the first to investigate both forward and backward processes of binarynetworks from the unified information perspective, which provides new insightinto the mechanism of network binarization. Comprehensive experiments withvarious network structures on CIFAR-10 and ImageNet datasets manifest that theproposed IR-Net can consistently outperform state-of-the-art quantizationmethods.

Forward and Backward Information Retention for Accurate Binary Neural Networks