English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper addresses the problem of selective classification for deep neuralnetworks, where a model is allowed to abstain from low-confidence predictionsto avoid potential errors. We focus on so-called post-hoc methods, whichreplace the confidence estimator of a given classifier without retraining ormodifying it, thus being practically appealing. Considering neural networkswith softmax outputs, our goal is to identify the best confidence estimatorthat can be computed directly from the unnormalized logits. This problem ismotivated by the intriguing observation in recent work that many classifiersappear to have a "broken" confidence estimator, in the sense that theirselective classification performance is much worse than what could be expectedby their corresponding accuracies. We perform an extensive experimental studyof many existing and proposed confidence estimators applied to 84 pretrainedImageNet classifiers available from popular repositories. Our results show thata simple $p$-norm normalization of the logits, followed by taking the maximumlogit as the confidence estimator, can lead to considerable gains in selectiveclassification performance, completely fixing the pathological behaviorobserved in many classifiers. As a consequence, the selective classificationperformance of any classifier becomes almost entirely determined by itscorresponding accuracy. Moreover, these results are shown to be consistentunder distribution shift. We also investigate why certain classifiers innatelyhave a good confidence estimator that apparently cannot be improved by post-hocmethods.

How to fix a broken confidence estimator: Evaluating post-hoc methods for selective classification with deep neural networks