English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

When deployed for risk-sensitive tasks, deep neural networks must include anuncertainty estimation mechanism. Here we examine the relationship between deeparchitectures and their respective training regimes, with their correspondingselective prediction and uncertainty estimation performance. We consider someof the most popular estimation performance metrics previously proposedincluding AUROC, ECE, AURC as well as coverage for selective accuracyconstraint. We present a novel and comprehensive study of selective predictionand the uncertainty estimation performance of 523 existing pretrained deepImageNet classifiers that are available in popular repositories. We identifynumerous and previously unknown factors that affect uncertainty estimation andexamine the relationships between the different metrics. We find thatdistillation-based training regimes consistently yield better uncertaintyestimations than other training schemes such as vanilla training, pretrainingon a larger dataset and adversarial training. Moreover, we find a subset of ViTmodels that outperform any other models in terms of uncertainty estimationperformance. For example, we discovered an unprecedented 99% top-1 selectiveaccuracy on ImageNet at 47% coverage (and 95% top-1 accuracy at 80%) for a ViTmodel, whereas a competing EfficientNet-V2-XL cannot obtain these accuracyconstraints at any level of coverage. Our companion paper, also published inICLR 2023 (A framework for benchmarking class-out-of-distribution detection andits application to ImageNet), examines the performance of these classifiers ina class-out-of-distribution setting.

What Can We Learn From The Selective Prediction And Uncertainty Estimation Performance Of 523 Imagenet Classifiers