English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Understanding the performance of machine learning (ML) models across diversedata distributions is critically important for reliable applications. Despiterecent empirical studies positing a near-perfect linear correlation betweenin-distribution (ID) and out-of-distribution (OOD) accuracies, we empiricallydemonstrate that this correlation is more nuanced under subpopulation shifts.Through rigorous experimentation and analysis across a variety of datasets,models, and training epochs, we demonstrate that OOD performance often has anonlinear correlation with ID performance in subpopulation shifts. Ourfindings, which contrast previous studies that have posited a linearcorrelation in model performance during distribution shifts, reveal a "moonshape" correlation (parabolic uptrend curve) between the test performance onthe majority subpopulation and the minority subpopulation. This non-trivialnonlinear correlation holds across model architectures, hyperparameters,training durations, and the imbalance between subpopulations. Furthermore, wefound that the nonlinearity of this "moon shape" is causally influenced by thedegree of spurious correlations in the training data. Our controlledexperiments show that stronger spurious correlation in the training datacreates more nonlinear performance correlation. We provide complementaryexperimental and theoretical analyses for this phenomenon, and discuss itsimplications for ML reliability and fairness. Our work highlights theimportance of understanding the nonlinear effects of model improvement onperformance in different subpopulations, and has the potential to inform thedevelopment of more equitable and responsible machine learning models.

Accuracy on the Curve: On the Nonlinear Correlation of ML Performance Between Data Subpopulations