English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We introduce Delay Pruning, a simple yet powerful technique to regularizedynamic Boltzmann machines (DyBM). The recently introduced DyBM provides aparticularly structured Boltzmann machine, as a generative model of amulti-dimensional time-series. This Boltzmann machine can have infinitely manylayers of units but allows exact inference and learning based on itsbiologically motivated structure. DyBM uses the idea of conduction delays inthe form of fixed length first-in first-out (FIFO) queues, with a neuronconnected to another via this FIFO queue, and spikes from a pre-synaptic neurontravel along the queue to the post-synaptic neuron with a constant period ofdelay. Here, we present Delay Pruning as a mechanism to prune the lengths ofthe FIFO queues (making them zero) by setting some delay lengths to one with afixed probability, and finally selecting the best performing model with fixeddelays. The uniqueness of structure and a non-sampling based learning rule inDyBM, make the application of previously proposed regularization techniqueslike Dropout or DropConnect difficult, leading to poor generalization. First,we evaluate the performance of Delay Pruning to let DyBM learn amultidimensional temporal sequence generated by a Markov chain. Finally, weshow the effectiveness of delay pruning in learning high dimensional sequencesusing the moving MNIST dataset, and compare it with Dropout and DropConnectmethods.

Regularized Dynamic Boltzmann Machine with Delay Pruning for Unsupervised Learning of Temporal Sequences