English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Artificial neural networks (ANNs) trained using backpropagation are powerfullearning architectures that have achieved state-of-the-art performance invarious benchmarks. Significant effort has been devoted to developing customsilicon devices to accelerate inference in ANNs. Accelerating the trainingphase, however, has attracted relatively little attention. In this paper, wedescribe a hardware-efficient on-line learning technique for feedforwardmulti-layer ANNs that is based on pipelined backpropagation. Learning isperformed in parallel with inference in the forward pass, removing the need foran explicit backward pass and requiring no extra weight lookup. By using binarystate variables in the feedforward network and ternary errors intruncated-error backpropagation, the need for any multiplications in theforward and backward passes is removed, and memory requirements for thepipelining are drastically reduced. Further reduction in addition operationsowing to the sparsity in the forward neural and backpropagating error signalpaths contributes to highly efficient hardware implementation. Forproof-of-concept validation, we demonstrate on-line learning of MNISThandwritten digit classification on a Spartan 6 FPGA interfacing with anexternal 1Gb DDR2 DRAM, that shows small degradation in test error performancecompared to an equivalently sized binary ANN trained off-line using standardback-propagation and exact errors. Our results highlight an attractive synergybetween pipelined backpropagation and binary-state networks in substantiallyreducing computation and memory requirements, making pipelined on-line learningpractical in deep networks.

Hardware-efficient on-line learning through pipelined truncated-error backpropagation in binary-state networks