English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Restricted Boltzmann machines (RBMs) and their extensions, called'deep-belief networks', are powerful neural networks that have foundapplications in the fields of machine learning and artificial intelligence. Thestandard way to training these models resorts to an iterative unsupervisedprocedure based on Gibbs sampling, called 'contrastive divergence' (CD), andadditional supervised tuning via back-propagation. However, this procedure hasbeen shown not to follow any gradient and can lead to suboptimal solutions. Inthis paper, we show an efficient alternative to CD by means of simulations ofdigital memcomputing machines (DMMs). We test our approach on patternrecognition using a modified version of the MNIST data set. DMMs sampleeffectively the vast phase space given by the model distribution of the RBM,and provide a very good approximation close to the optimum. This efficientsearch significantly reduces the number of pretraining iterations necessary toachieve a given level of accuracy, as well as a total performance gain over CD.In fact, the acceleration of pretraining achieved by simulating DMMs iscomparable to, in number of iterations, the recently reported hardwareapplication of the quantum annealing method on the same network and data set.Notably, however, DMMs perform far better than the reported quantum annealingresults in terms of quality of the training. We also compare our method toadvances in supervised training, like batch-normalization and rectifiers, thatwork to reduce the advantage of pretraining. We find that the memcomputingmethod still maintains a quality advantage ($>1\%$ in accuracy, and a $20\%$reduction in error rate) over these approaches. Furthermore, our method isagnostic about the connectivity of the network. Therefore, it can be extendedto train full Boltzmann machines, and even deep networks at once.

Accelerating Deep Learning with Memcomputing