English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Covariate shift has been shown to sharply degrade both predictive accuracyand the calibration of uncertainty estimates for deep learning models. This isworrying, because covariate shift is prevalent in a wide range of real worlddeployment settings. However, in this paper, we note that frequently thereexists the potential to access small unlabeled batches of the shifted data justbefore prediction time. This interesting observation enables a simple butsurprisingly effective method which we call prediction-time batchnormalization, which significantly improves model accuracy and calibrationunder covariate shift. Using this one line code change, we achievestate-of-the-art on recent covariate shift benchmarks and an mCE of 60.28\% onthe challenging ImageNet-C dataset; to our knowledge, this is the best resultfor any model that does not incorporate additional data augmentation ormodification of the training pipeline. We show that prediction-time batchnormalization provides complementary benefits to existing state-of-the-artapproaches for improving robustness (e.g. deep ensembles) and combining the twofurther improves performance. Our findings are supported by detailedmeasurements of the effect of this strategy on model behavior across rigorousablations on various dataset modalities. However, the method has mixed resultswhen used alongside pre-training, and does not seem to perform as well undermore natural types of dataset shift, and is therefore worthy of additionalstudy. We include links to the data in our figures to improve reproducibility,including a Python notebooks that can be run to easily modify our analysis athttps://colab.research.google.com/drive/11N0wDZnMQQuLrRwRoumDCrhSaIhkqjof.

Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift