English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Bayesian optimization is a highly efficient approach to optimizing objectivefunctions which are expensive to query. These objectives are typicallyrepresented by Gaussian process (GP) surrogate models which are easy tooptimize and support exact inference. While standard GP surrogates have beenwell-established in Bayesian optimization, Bayesian neural networks (BNNs) haverecently become practical function approximators, with many benefits overstandard GPs such as the ability to naturally handle non-stationarity and learnrepresentations for high-dimensional data. In this paper, we study BNNs asalternatives to standard GP surrogates for optimization. We consider a varietyof approximate inference procedures for finite-width BNNs, includinghigh-quality Hamiltonian Monte Carlo, low-cost stochastic MCMC, and heuristicssuch as deep ensembles. We also consider infinite-width BNNs and partiallystochastic models such as deep kernel learning. We evaluate this collection ofsurrogate models on diverse problems with varying dimensionality, number ofobjectives, non-stationarity, and discrete and continuous inputs. We find: (i)the ranking of methods is highly problem dependent, suggesting the need fortailored inductive biases; (ii) HMC is the most successful approximateinference procedure for fully stochastic BNNs; (iii) full stochasticity may beunnecessary as deep kernel learning is relatively competitive; (iv)infinite-width BNNs are particularly promising, especially in high dimensions.

A Study of Bayesian Neural Network Surrogates for Bayesian Optimization