English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Proper confidence calibration of deep neural networks is essential forreliable predictions in safety-critical tasks. Miscalibration can lead to modelover-confidence and/or under-confidence; i.e., the model's confidence in itsprediction can be greater or less than the model's accuracy. Recent studieshave highlighted the over-confidence issue by introducing calibrationtechniques and demonstrated success on various tasks. However, miscalibrationthrough under-confidence has not yet to receive much attention. In this paper,we address the necessity of paying attention to the under-confidence issue. Wefirst introduce a novel metric, a miscalibration score, to identify the overalland class-wise calibration status, including being over or under-confident. Ourproposed metric reveals the pitfalls of existing calibration techniques, wherethey often overly calibrate the model and worsen under-confident predictions.Then we utilize the class-wise miscalibration score as a proxy to design acalibration technique that can tackle both over and under-confidence. We reportextensive experiments that show our proposed methods substantiallyoutperforming existing calibration techniques. We also validate our proposedcalibration technique on an automatic failure detection task with arisk-coverage curve, reporting that our methods improve failure detection aswell as trustworthiness of the model. The code are available at\url{https://github.com/AoShuang92/miscalibration_TS}.

Two Sides of Miscalibration: Identifying Over and Under-Confidence Prediction for Network Calibration