English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We present a theoretical and experimental investigation of the quantizationproblem for artificial neural networks. We provide a mathematical definition ofquantized neural networks and analyze their approximation capabilities, showingin particular that any Lipschitz-continuous map defined on a hypercube can beuniformly approximated by a quantized neural network. We then focus on theregularization effect of additive noise on the arguments of multi-stepfunctions inherent to the quantization of continuous variables. In particular,when the expectation operator is applied to a non-differentiable multi-steprandom function, and if the underlying probability density is differentiable(in either classical or weak sense), then a differentiable function isretrieved, with explicit bounds on its Lipschitz constant. Based on theseresults, we propose a novel gradient-based training algorithm for quantizedneural networks that generalizes the straight-through estimator, acting onnoise applied to the network's parameters. We evaluate our algorithm on theCIFAR-10 and ImageNet image classification benchmarks, showing state-of-the-artperformance on AlexNet and MobileNetV2 for ternary networks.

Additive Noise Annealing and Approximation Properties of Quantized Neural Networks