English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Sparse deep neural networks have proven to be efficient for predictive modelbuilding in large-scale studies. Although several works have studiedtheoretical and numerical properties of sparse neural architectures, they haveprimarily focused on the edge selection. Sparsity through edge selection mightbe intuitively appealing; however, it does not necessarily reduce thestructural complexity of a network. Instead pruning excessive nodes leads to astructurally sparse network with significant computational speedup duringinference. To this end, we propose a Bayesian sparse solution usingspike-and-slab Gaussian priors to allow for automatic node selection duringtraining. The use of spike-and-slab prior alleviates the need of an ad-hocthresholding rule for pruning. In addition, we adopt a variational Bayesapproach to circumvent the computational challenges of traditional Markov ChainMonte Carlo (MCMC) implementation. In the context of node selection, weestablish the fundamental result of variational posterior consistency togetherwith the characterization of prior parameters. In contrast to the previousworks, our theoretical development relaxes the assumptions of the equal numberof nodes and uniform bounds on all network weights, thereby accommodatingsparse networks with layer-dependent node structures or coefficient bounds.With a layer-wise characterization of prior inclusion probabilities, we discussthe optimal contraction rates of the variational posterior. We empiricallydemonstrate that our proposed approach outperforms the edge selection method incomputational complexity with similar or better predictive performance. Ourexperimental evidence further substantiates that our theoretical workfacilitates layer-wise optimal node recovery.

Layer Adaptive Node Selection in Bayesian Neural Networks: Statistical Guarantees and Implementation Details