English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This work aims to analyze standard evaluation practices adopted by theresearch community when assessing chest x-ray classifiers, particularlyfocusing on the impact of class imbalance in such appraisals. Our analysisconsiders a comprehensive definition of model performance, covering not onlydiscriminative performance but also model calibration, a topic of research thathas received increasing attention during the last years within the machinelearning community. Firstly, we conducted a literature study to analyze commonscientific practices and confirmed that: (1) even when dealing with highlyimbalanced datasets, the community tends to use metrics that are dominated bythe majority class; and (2) it is still uncommon to include calibration studiesfor chest x-ray classifiers, albeit its importance in the context ofhealthcare. Secondly, we perform a systematic experiment on two major chestx-ray datasets to explore the behavior of several performance metrics underdifferent class ratios and show that widely adopted metrics can conceal theperformance in the minority class. Finally, we recommend the inclusion ofcomplementary metrics to better reflect the system's performance in suchscenarios. Our study indicates that current evaluation practices adopted by theresearch community for chest x-ray computer-aided diagnosis systems may notreflect their performance in real clinical scenarios, and suggest alternativesto improve this situation.

Impact of class imbalance on chest x-ray classifiers: towards better evaluation practices for discrimination and calibration performance