English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Restricted Boltzmann Machine (RBM) is a bipartite graphical model that isused as the building block in energy-based deep generative models. Due tonumerical stability and quantifiability of the likelihood, RBM is commonly usedwith Bernoulli units. Here, we consider an alternative member of exponentialfamily RBM with leaky rectified linear units -- called leaky RBM. We firststudy the joint and marginal distributions of leaky RBM under differentleakiness, which provides us important insights by connecting the leaky RBMmodel and truncated Gaussian distributions. The connection leads us to a simpleyet efficient method for sampling from this model, where the basic idea is toanneal the leakiness rather than the energy; -- i.e., start from a fullyGaussian/Linear unit and gradually decrease the leakiness over iterations. Thisserves as an alternative to the annealing of the temperature parameter andenables numerical estimation of the likelihood that are more efficient and moreaccurate than the commonly used annealed importance sampling (AIS). We furtherdemonstrate that the proposed sampling algorithm enjoys faster mixing propertythan contrastive divergence algorithm, which benefits the training without anyadditional computational cost.

Annealing Gaussian into ReLU: a New Sampling Strategy for Leaky-ReLU RBM