English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Binary Neural Networks (BNNs) have been garnering interest thanks to theircompute cost reduction and memory savings. However, BNNs suffer fromperformance degradation mainly due to the gradient mismatch caused bybinarizing activations. Previous works tried to address the gradient mismatchproblem by reducing the discrepancy between activation functions used atforward pass and its differentiable approximation used at backward pass, whichis an indirect measure. In this work, we use the gradient of smoothed lossfunction to better estimate the gradient mismatch in quantized neural network.Analysis using the gradient mismatch estimator indicates that using higherprecision for activation is more effective than modifying the differentiableapproximation of activation function. Based on the observation, we propose anew training scheme for binary activation networks called BinaryDuo in whichtwo binary activations are coupled into a ternary activation during training.Experimental results show that BinaryDuo outperforms state-of-the-art BNNs onvarious benchmarks with the same amount of parameters and computing cost.

BinaryDuo: Reducing Gradient Mismatch in Binary Activation Network by Coupling Binary Activations