English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The "cold posterior effect" (CPE) in Bayesian deep learning describes theuncomforting observation that the predictive performance of Bayesian neuralnetworks can be significantly improved if the Bayes posterior is artificiallysharpened using a temperature parameter T<1. The CPE is problematic in theoryand practice and since the effect was identified many researchers have proposedhypotheses to explain the phenomenon. However, despite this intensive researcheffort the effect remains poorly understood. In this work we provide novel andnuanced evidence relevant to existing explanations for the cold posterioreffect, disentangling three hypotheses: 1. The dataset curation hypothesis ofAitchison (2020): we show empirically that the CPE does not arise in a realcurated data set but can be produced in a controlled experiment with varyingcuration strength. 2. The data augmentation hypothesis of Izmailov et al.(2021) and Fortuin et al. (2021): we show empirically that data augmentation issufficient but not necessary for the CPE to be present. 3. The bad priorhypothesis of Wenzel et al. (2020): we use a simple experiment evaluating therelative importance of the prior and the likelihood, strongly linking the CPEto the prior. Our results demonstrate how the CPE can arise in isolation fromsynthetic curation, data augmentation, and bad priors. Cold posteriors observed"in the wild" are therefore unlikely to arise from a single simple cause; as aresult, we do not expect a simple "fix" for cold posteriors.

Disentangling the Roles of Curation, Data-Augmentation and the Prior in the Cold Posterior Effect