English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Backward propagation of errors (backpropagation) is a method to minimizeobjective functions (e.g., loss functions) of deep neural networks byidentifying optimal sets of weights and biases. Imposing constraints on weightprecision is often required to alleviate prohibitive workloads on hardware.Despite the remarkable success of backpropagation, the algorithm itself is notcapable of considering such constraints unless additional algorithms areapplied simultaneously. To address this issue, we propose the constrainedbackpropagation (CBP) algorithm based on a pseudo-Lagrange multiplier method toobtain the optimal set of weights that satisfy a given set of constraints. Thedefining characteristic of the proposed CBP algorithm is the utilization of aLagrangian function (loss function plus constraint function) as its objectivefunction. We considered various types of constraints--binary, ternary, one-bitshift, and two-bit shift weight constraints. As a post-training method, CBPapplied to AlexNet, ResNet-18, ResNet-50, and GoogLeNet on ImageNet, which werepre-trained using the conventional backpropagation. For all cases, the proposedalgorithm outperforms the state-of-the-art methods on ImageNet, e.g., 66.6%,74.4%, and 64.0% top-1 accuracy for ResNet-18, ResNet-50, and GoogLeNet withbinary weights, respectively. This highlights CBP as a learning algorithm toaddress diverse constraints with the minimal performance loss by employingappropriate constraint functions.

CBP: Backpropagation with constraint on weight precision using a pseudo-Lagrange multiplier method