English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Averaging predictions of a deep ensemble of networks is apopular andeffective method to improve predictive performance andcalibration in variousbenchmarks and Kaggle competitions. However, theruntime and training cost ofdeep ensembles grow linearly with the size ofthe ensemble, making themunsuitable for many applications. Averagingensemble weights instead ofpredictions circumvents this disadvantageduring inference and is typicallyapplied to intermediate checkpoints ofa model to reduce training cost. Albeiteffective, only few works haveimproved the understanding and the performance ofweight averaging.Here, we revisit this approach and show that a simple weightfusion (WF)strategy can lead to a significantly improved predictive performanceandcalibration. We describe what prerequisites the weights must meet interms ofweight space, functional space and loss. Furthermore, we presenta new testmethod (called oracle test) to measure the functional spacebetween weights. Wedemonstrate the versatility of our WF strategy acrossstate of the artsegmentation CNNs and Transformers as well as real worlddatasets such asBDD100K and Cityscapes. We compare WF with similarapproaches and show oursuperiority for in- and out-of-distribution datain terms of predictiveperformance and calibration.

Improving Predictive Performance and Calibration by Weight Fusion in Semantic Segmentation