English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Existing works show that although modern neural networks achieve remarkablegeneralization performance on the in-distribution (ID) dataset, the accuracydrops significantly on the out-of-distribution (OOD) datasets\cite{recht2018cifar, recht2019imagenet}. To understand why a variety of modelsconsistently make more mistakes in the OOD datasets, we propose a new metric toquantify the difficulty of the test images (either ID or OOD) that depends onthe interaction of the training dataset and the model. In particular, weintroduce \textit{confusion score} as a label-free measure of image difficultywhich quantifies the amount of disagreement on a given test image based on theclass conditional probabilities estimated by an ensemble of trained models.Using the confusion score, we investigate CIFAR-10 and its OOD derivatives.Next, by partitioning test and OOD datasets via their confusion scores, wepredict the relationship between ID and OOD accuracies for variousarchitectures. This allows us to obtain an estimator of the OOD accuracy of agiven model only using ID test labels. Our observations indicate that thebiggest contribution to the accuracy drop comes from images with high confusionscores. Upon further inspection, we report on the nature of the misclassifiedimages grouped by their confusion scores: \textit{(i)} images with highconfusion scores contain \textit{weak spurious correlations} that appear inmultiple classes in the training data and lack clear \textit{class-specificfeatures}, and \textit{(ii)} images with low confusion scores exhibit spuriouscorrelations that belong to another class, namely \textit{class-specificspurious correlations}.

Understanding out-of-distribution accuracies through quantifying difficulty of test samples