English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

An ensemble of neural networks is known to be more robust and accurate thanan individual network, however usually with linearly-increased cost in bothtraining and testing. In this work, we propose a two-stage method to learnSparse Structured Ensembles (SSEs) for neural networks. In the first stage, werun SG-MCMC with group sparse priors to draw an ensemble of samples from theposterior distribution of network parameters. In the second stage, we applyweight-pruning to each sampled network and then perform retraining over theremained connections. In this way of learning SSEs with SG-MCMC and pruning, wenot only achieve high prediction accuracy since SG-MCMC enhances exploration ofthe model-parameter space, but also reduce memory and computation costsignificantly in both training and testing of NN ensembles. This is thoroughlyevaluated in the experiments of learning SSE ensembles of both FNNs and LSTMs.For example, in LSTM based language modeling (LM), we obtain 21% relativereduction in LM perplexity by learning a SSE of 4 large LSTM models, which hasonly 30% of model parameters and 70% of computations in total, as compared tothe baseline large LSTM LM. To the best of our knowledge, this work representsthe first methodology and empirical study of integrating SG-MCMC, group sparseprior and network pruning together for learning NN ensembles.

Learning Sparse Structured Ensembles with SG-MCMC and Network Pruning