English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this thesis, we disentangle the generalized Gauss-Newton and approximateinference for Bayesian deep learning. The generalized Gauss-Newton method is anoptimization method that is used in several popular Bayesian deep learningalgorithms. Algorithms that combine the Gauss-Newton method with the Laplaceand Gaussian variational approximation have recently led to state-of-the-artresults in Bayesian deep learning. While the Laplace and Gaussian variationalapproximation have been studied extensively, their interplay with theGauss-Newton method remains unclear. Recent criticism of priors and posteriorapproximations in Bayesian deep learning further urges the need for a deeperunderstanding of practical algorithms. The individual analysis of theGauss-Newton method and Laplace and Gaussian variational approximations forneural networks provides both theoretical insight and new practical algorithms.We find that the Gauss-Newton method simplifies the underlying probabilisticmodel significantly. In particular, the combination of the Gauss-Newton methodwith approximate inference can be cast as inference in a linear or Gaussianprocess model. The Laplace and Gaussian variational approximation cansubsequently provide a posterior approximation to these simplified models. Thisnew disentangled understanding of recent Bayesian deep learning algorithms alsoleads to new methods: first, the connection to Gaussian processes enables newfunction-space inference algorithms. Second, we present a marginal likelihoodapproximation of the underlying probabilistic model to tune neural networkhyperparameters. Finally, the identified underlying models lead to differentmethods to compute predictive distributions. In fact, we find that theseprediction methods for Bayesian neural networks often work better than thedefault choice and solve a common issue with the Laplace approximation.

Disentangling the Gauss-Newton Method and Approximate Inference for Neural Networks